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I .  INTRODUCTION 

A.  SYSTEM  INTRODUCTION 

The  Smart  Phone  Assisted  Rapid  Communication  and 
Control  System  (SPARCCS)  is  a  system  in  development  in  the 
Computer  Science  Department  of  the  Naval  Postgraduate 
School.  The  system  is  designed  to  enable  rapid 
communications  in  the  military  field  or  during  civilian 
emergency  operations  by  facilitating  teams  with  mobile 
devices  to  capture  images  and  share  information  with  one 
another  and  with  a  central  command  and  control  center. 

The  system  is  currently  in  development.  The  initial 
architecture  was  developed  by  Asche  and  Crews  (2012) 
enabling  a  smart  phone  to  communicate  with  the  server  and 
upload  information  and  images.  The  system  is  functional  at 
this  initial  stage  of  development. 

B .  PROBLEM  STATEMENT 

The  SPARCCS  system  has  had  no  formal  testing  since  its 
inception.  As  a  result,  the  system  is  at  risk  while  it 
enters  into  later  stages  of  development.  Problems  in 
systems  discovered  late  in  the  development  stage  are 
exponentially  more  expensive  to  fix  in  terms  of  time, 
money,  and  resources.  As  a  result,  the  SPARCCS  system 
requires  comprehensive  testing  at  its  current  stage  of 
development  to  help  eliminate  problems  that  may  affect  its 
reliability,  availability,  and  safety. 
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c. 


RESEARCH  DEPTH  AND  QUESTIONS 


The  research  presented  in  this  thesis  will  be  the 
initial  testing  of  the  system,  based  on  the  preliminary 
development  of  the  system  as  presented  by  Asche  and  Crews 
(March,  2012)  .  At  the  time  of  the  research  in  this  thesis, 
the  SPARCCS  system  has  a  preliminary  foundation  developed, 
including  the  basic  web-based  command  center  and  the 
Android  application  for  the  mobile  handheld  devices  that 
runs  on  Android  smart  phones  and  Android  tablets. 


Figure  1.  Test  Domain  of  the  Thesis 

This  thesis  will  test  the  current  SPARCCS  system  with 

various  wireless  technologies  such  as  WIFI,  BGAN,  and  Wave 

Relay,  as  depicted  in  Figure  1  above.  A  variety  of  handheld 

devices  such  as  smart  phones  and  tablet  PCs  will  be  tested 

to  demonstrate  the  application' s  performance  on  various 
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platforms.  Additionally,  several  versions  of  the  Android 
operating  system,  ranging  from  version  2.2  through  version 
4.0,  will  be  tested. 

The  primary  research  questions  for  this  thesis  are: 

1.  What  is  an  appropriate  testing  plan  for  the  SPARCCS 
system  and  what  are  the  results  of  the  tests  in  the 
testing  plan  that  can  be  used  to  evaluate  and  improve 
the  system? 

2 .  How  can  tests  be  used  to  evaluate  the  architecture 
and  basic  functionality  of  the  SPARCCS  system? 

3.  What  tests  are  necessary  to  evaluate  the  SPARCCS 
and  what  are  the  results  of  those  tests? 

4 .  What  future  tests  can  be  used  on  the  system  in  the 
system' s  later  development  that  would  relate  to  the 
original  test  plan? 

5.  How  can  the  SPARCCS  system  be  analyzed  to 
facilitate  proper  testing,  verification  and 
validation? 

D .  SYSTEM  OVERVIEW 

The  overall  system  will  consist  of  mobile  users  in  a 
WIFI  cloud  with  a  team  leader  and  a  set  of  team  members, 
all  with  smart  phones.  The  team  leaders  will  have  their 
smart  phones  networked  with  the  team  members  in  the  cloud. 
The  team  leaders  will  be  able  to  connect  to  a  satellite  via 
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a  BGAN  connection  and  to  a  local  server  via  technologies 
such  as  Persistent  Systems  Wave  Relay  radio-based  router. 

The  server  will  communicate  information  to  a  command 
and  control  (C2)  center.  Additionally,  a  small  UAV  may  be 
part  of  the  field  teams  to  facilitate  airborne  image  taking 
and  scanning  which  will  give  a  broader  picture  of  the 

operational  area  as  well  as  communications  relay.  Each 
smart  phone  will  have  a  small  database.  Each  database  will 
stay  in  synch  with  the  master  phone  of  the  team  leader.  The 
team  leader  will  then  be  able  to  send  comprehensive 

situational  information  to  the  C2  center.  The  team  leader 
will  also  be  able  to  send  information  from  the  UAV  to  the 
C2  center. 

The  overall  goal  of  the  system  is  to  be  able  to 
provide  real-time  information  to  a  C2  center  as  to 

situational  information  in  an  area  of  operation  to  foster 
command  and  team  situational  awareness.  This  will  promote 
rapid  information  transfer  that  will  facilitate  enhanced 
leadership  decision-making  and  will  help  reduce  operational 
decision  cycles.  Additionally,  communication  and  data  will 
be  organized  in  a  methodical  manner  facilitating  strong 

communications  and  data  exchange  to  promote  incident  and 
scene  management  and  awareness. 
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Figure  2.  DoD  OV-1  Diagram  of  the  SPARCCS  System 

Various  technologies  will  facilitate  the 
communications  cloud  around  the  teams.  The  first  technology 
is  WIFI,  which  will  be  provided  through  a  commercial 
wireless  service.  The  second  technology  is  a  satellite 
Internet  technology  known  as  BGAN  (Broadband  Global  Area 
Network) ,  which  is  a  proprietary  satellite  Internet 
service.  BGAN  uses  the  INMARSAT  satellite  constellation  to 
facilitate  Internet  connectivity. 

The  third  technology  that  will  be  tested  is  a  Wave 
Relay  system.  This  system  consists  of  proprietary  radio 
systems  that  act  as  both  bridges  and  wireless  hot  spots 
that  extend  the  range  of  WIFI  clouds  to  distant  teams  using 
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the  SPARCCS  system.  The  Wave  Relay  technology,  developed  by 
Persistent  Systems,  will  use  the  BGAN  technology  as  the 
back-haul  of  its  Internet  connectivity.  The  final 
technology  will  be  the  WIMAX  technology,  a  broadband 
wireless  communications  technology  that  acts  as  a  bridge 
for  wireless  communications. 

The  overall  goal  of  each  technology  is  to  sustain  and 
maintain  a  viable  WIFI  cloud,  with  Internet  reach-back, 
over  the  field  teams  to  facilitate  server  and  inter-team 
communications  at  all  times. 

E.  SPARCCS  SYSTEM  CONTEXT  DIAGRAM 

A  context  diagram  is  a  systems  engineering-based 
diagram  that  provides  an  overview  of  a  system  by 
decomposing  a  system  into  its  external  entities  and  the 
products  that  are  exchanged  between  the  system  and  those 
entities  (Kossiakoff  &  Sweet,  2011).  The  external  entities 
are  all  of  the  entities  that  affect  the  system  in  some  way, 
including  the  environment.  The  purpose  of  context  diagrams 
is  to  highlight  the  entities  that  will  interact  with  a 
system,  including  entities  that  provide  maintenance, 
updates,  and  streams  of  data.  Entities  are  any  external 
influence,  including  other  systems,  media,  and  forms  of 
support . 

Context  diagrams  enable  the  analysis  of  a  system  by 
identifying  the  sources  of  data  and  information  flows  as 
well  as  the  presence  of  power,  commands,  heat,  weather,  and 
so  forth.  Through  the  decomposition  of  these  entities,  a 
more  complete  picture  of  the  system  can  be  developed  so 
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that  the  testing  of  the  system  can  be  optimized  (Biemer, 
2010) .  The  context  diagram  of  the  SPARCCS  system  is 
presented  in  Figure  2 . 


Figure  3.  Context  Diagram  of  the  SPARCCS  System 

As  can  be  seen  in  the  context  diagram,  there  are 
several  key  entities  that  interact  with  the  SPARCCS  system. 
Each  entity  must  be  considered  in  the  testing  program  and 
the  interactive  products  between  the  SPARCCS  and  the 
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program  to  ensure  the  viability,  accuracy,  and  stability  of 
those  interactions. 

F .  CHAPTER  SUMMARY 

The  test  plan  and  implementation  of  the  plan  as  well 
as  the  formal  analysis  of  the  system  in  this  thesis  will 
ensure  the  proper  operation,  verification,  and  validation 
of  the  SPARCCS  system,  which  has  currently  been  partially 
developed  and  is  ready  for  initial  architectural  and 
systems  testing.  The  SPARCCS  testing  will  focus  on 
performance  and  deployment  testing  to  ensure  system 
functionality  in  its  current  stage  of  development.  The 
testing  will  also  focus  on  the  usability  of  the  system 
including  installation  and  user  interface  issues  and 
capabilities . 

G.  THESIS  STRUCTURE 

The  structure  of  this  thesis  roughly  corresponds  to  a 
software  systems  engineering  testing  plan  that  would  be 
performed  in  the  operational  environment  in  most  DoD 
commercial  and  government  systems  development. 

Chapter  II  describes  the  testing  methodologies  in 

detail  and  relates  them  to  current  best  practices  in  the 
literature.  It  details  wireless  testing  problems,  the 
systems  engineering  testing  paradigms  for  software  testing, 
test  design,  test  planning,  and  systems  test  partitioning. 

The  concept  of  a  controlled  laboratory  is  also  discussed. 

In  addition,  the  principles  of  integration,  interface  and 

functional  testing  are  explored,  as  are  the  concepts  of 
performance  and  deployment  testing,  which  are  the  central 
focus  of  the  thesis.  Finally,  the  principles  of  wireless 
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hardware  and  software  testing  are  covered  to  round  out  the 
discussion.  The  chapter  concludes  with  a  brief  on  when  to 
stop  testing,  an  important  concept  in  all  engineering  and 
systems  testing  programs. 

Chapter  III  develops  an  integrated  test  plan  for  the 
SPARCCS  system.  It  begins  with  a  thorough  reguirements 
analysis  for  the  specific  scope  of  the  system  that  will  be 
tested  in  this  thesis.  The  requirements  are  placed  in  a 
VCRM  matrix,  which  is  a  requirements  engineering  tool  for 
the  management  of  system  requirements.  The  test  plan 
approach  is  then  developed  to  demonstrate  the  comprehensive 
performance  testing  approach,  the  levels  of  testing  and  the 
various  test  environments.  The  chapter  includes  the 
qualification  plan  to  ensure  that  the  test  plan  and 
sequences  are  accurate  and  appropriate  for  the  levels  and 
scope  of  the  required  SPARCCS  testing.  The  chapter  develops 
the  specific  test  cases,  including  the  indoor  and  outdoor 
controlled  testing  as  well  as  the  field  test  plans  for  all 
of  the  required  wireless  technology.  The  chapter  also 
describes  in  detail  the  test  equipment  and  the  wireless 
technology  to  be  tested  in  the  field  tests.  This  chapter 
rounds  out  the  description  of  the  testing  and  equipment. 

Chapter  IV  describes  the  results  of  the  indoor  and 
outdoor  controlled  testing  and  the  results  of  the  field 
testing  of  all  of  the  technologies.  It  provides  selected 
raw  and  analyzed  data  and  explains  the  results  of  the  tests 
in  a  systematic  manner. 

Chapter  V  provides  the  conclusions  of  the  testing 
program.  These  are  the  high  level  results  that  will  affect 
the  continual  development  of  the  system  as  well  as  the 
operational  plans  of  the  system.  These  results  will  assist 
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in  improving  the  system  as  it  moves  from  the  development  to 
the  implementation  phase  of  its  project  cycle.  The  chapter 
provides  final  recommendations  that  can  be  applied  to  the 
system  development  and  operational  implementation  to 
improve  the  system.  In  addition,  the  chapter  provides  areas 
for  future  testing  and  new  technologies  or  new 
configurations  of  tested  technologies  that  may  help  improve 
the  system. 
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II.  SPARCCS  SYSTEM  TESTING  METHODOLOGY 


This  chapter  describes  the  SPARCCS  system  testing 
methodologies  in  detail  and  relates  them  to  current  best 
practices  in  the  literature.  It  details  wireless  testing 
problems,  the  systems  engineering  testing  paradigms  for 
software  testing,  test  design,  test  planning  and  systems 
test  partitioning.  The  concept  of  a  controlled  laboratory 
is  also  discussed.  In  addition,  the  principles  of 
integration,  interface  and  functional  testing  are  explored, 
as  are  the  concepts  of  performance  and  deployment  testing, 
which  are  the  central  focus  of  the  thesis.  Finally,  the 
principles  of  wireless  hardware  and  software  testing  are 
covered.  The  chapter  concludes  with  a  brief  on  when  to  stop 
testing,  an  important  concept  in  all  engineering  and 
systems  testing  programs. 

A.  INTRODUCTION  TO  SPARCCS  SYSTEMS  TESTING 

Testing  is  critical  in  any  system,  but  especially  in  a 
wireless  system  such  as  SPARCCS  due  to  the  complex  software 
and  data  sensitive  nature  of  the  wireless  system  in 
addition  to  its  integrated  communications  systems  backbone 
(Viana,  Maag,  and  Zaidi,  2011)  .  In  many  complex  software 
intensive  systems,  major  problems  can  manifest  at  any  point 
in  its  modules  and  systems,  and  therefore  comprehensive 
testing  is  crucial  (Jorgensen,  2002). 

The  SPARCCS  system  has  several  distinct  sub-systems 
that  need  to  be  integrated  and  as  such,  architectural, 
systems,  and  integration  testing  are  reguired  to  ensure 
that  the  system  has  a  reliable  construct.  The  SPARCCS 
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system  has  a  wide  array  of  modules  and  sub-systems,  making 
the  integration  of  them  critical  as  well,  which  must  be 
validated  by  testing. 

Through  rigorous  testing,  developers  of  future  SPARCCS 
modules  can  eliminate  the  interface,  architectural,  and 
interoperability  faults  and  errors,  and  thus  future  systems 
development  will  go  smoother  and  be  more  reliable. 

Wireless  architectural  and  interoperability  problems 
can  include  (Bell,  Jung,  &  Krishnakumar ,  2010;  Nguyen, 
Waeselnck,  &  Riviere,  2008;  &  Zhifang,  Bin,  &  Xiaopeng, 

2010)  : 

1.  Incorrect  interrupt  handling 

2 .  I/O  Timing 

3.  Call  to  wrong/nonexistent  procedure 

4.  Variable  mismatch 

5.  Parameter  mismatch 

6.  Incompatible  data  types 

7.  Hardware  issues  incompatibilities 

8.  Wireless  communications  incompatibilities 

9.  Protocol  programming  issues  (TCI/IP) 

10.  Computational  inconsistencies 

11.  GPS  issues 

12.  Internet  issues 

13.  System  interface  issues 

14.  Wireless  protocol  issues 

15.  User  interface  issues 

As  can  be  seen  in  the  above  list,  major  issues  can 
take  place  at  the  architectural  and  interoperability  level 
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(Giadrosich,  1995).  Through  rigorous  testing,  these  issues 
can  be  detected  and  rectified  before  the  system  proceeds 
through  its  future  development. 

The  SPARCCS  system  is  in  its  development  phase.  The 
overall  basic  architecture  has  been  developed.  The  testing 
plan  will  focus  on  the  initial  architecture  to  ensure  that 
the  architecture  is  viable  and  will  function  reliably  as 
the  foundation  for  future  SPARCCS  modules  and  systems.  The 
testing  plan  will  be  developed  and  executed  to  ensure  that 
the  architectural  foundation  and  current  development  of  the 
system  is  technologically  and  architecturally  sound  for 
future  development. 

The  testing  process  for  the  SPARCCS  will  be  divided 
into  three  major  components:  hardware  testing,  software 
testing,  and  systems  testing.  Each  area  of  testing  is 
detailed  in  the  sections  below. 

The  testing  of  the  SPARCCS  system  will  be  based  on 
solid  testing  principles  founded  upon  the  systems 
engineering  and  computer  science  disciplines.  By  combining 
systems  engineering  and  computer  science  testing 
principles,  a  more  precise  and  well-rounded  set  of  test 
results  can  be  obtained  to  ensure  real  world  accuracy  and 
applicability  of  the  test  results  (Kasser,  2007). 

B.  SYSTEMS  ENGINEERING  BASED  TESTING 

System  testing  is  critical  to  ensure  that  the  system 
functions  properly,  reliably  and  safely  in  terms  of  the 
system  requirements  and  the  system  contract,  proper  safety 
criteria,  and  reliability  requirements  (Giadrosich,  1995). 
Testing  helps  reduce  risks  and  helps  assure  proper  human 
factors  engineering  in  the  system.  It  also  helps  to  detect 
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problems  and  issues  with  the  system,  so  that  they  can  be 
rectified  as  early  in  the  project  as  possible.  Management 
of  risks  through  testing  helps  in  developing,  producing, 
operating  and  sustaining  the  system  and  its  capabilities. 
Testing  helps  ensure  that  system  components  meet  their 
purpose  and  are  in  full  compliance  with  their 
specifications  (Jorgensen,  2002).  Testing  helps  avoid  system 
problems  in  the  short  term  and  long  run. 

1.  Test  Design 

The  test  design  seeks  to  ensure  the  design  fulfills  a 
need  by  the  user/customer  of  the  system.  The  user  expects 
robust  wireless  communications  performance  from  a  system 
that  is  reliable,  available,  and  that  will  perform  well  in 
complex  environments.  The  functional  flows  of  the  system 
design  demonstrate  the  key  subsystem  responsibilities,  the 
interactions  between  the  subsystems  that  provide  critical 
test  points  for  the  testing  plan  (Kossiakoff  &  Sweet, 
2011)  .  Thus,  functional  analysis  must  be  conducted  on  the 
system.  For  the  SPARCCS  system,  functional  analysis  was 
conducted  and  the  previous  chapter  includes  the  resulting 
analysis  and  diagrams  of  the  system,  as  an  introduction  to 
the  system. 

System  interfaces,  modules,  and  subsystems  can  be 
tested  more  thoroughly  and  properly  when  design  and  testing 
plans  are  created  in  a  systematic,  well  defined  manner 
(Kossiakoff  and  Sweet,  2011)  .  The  interfaces,  modules  and 
subsystems  are  the  major  points  of  failure  in  complex 
systems,  and  therefore  the  designs  of  the  system  must  be 
carefully  developed  by  systems  engineers  and  tested  in  the 
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same  systematic  manner  with  a  careful  consideration  of  the 
design  and  interactions  of  the  interfaces,  modules,  and 
subsystems  (Wasson,  2006) . 

2 .  Test  Plans  and  Design 

Test  plans  flow  in  parallel  from  the  system  test 
design.  Multiple  test  plans  can  be  developed  and  conducted 
simultaneously  to  ensure  timeliness  of  the  testing  and  to 
ensure  that  the  testing  scope  and  timeline  keep  the  project 
on  the  critical  path.  From  the  test  event  timeline  the  test 
procedures  are  developed  (Biemer,  2010) . 

As  can  be  seen,  both  the  design  and  the  test  plan  flow 
in  a  hierarchical  manner,  with  the  test  plan  elements 
referring  back  to  the  design  plan  elements  (Kossiakoff  & 
Sweet,  2011)  .  This  parallel  nature  is  critical  in  the 
testing  of  all  system  of  systems  to  ensure  that  all  aspects 
of  the  design  are  tested  thoroughly  and  completely.  As  can 
be  seen,  this  parallel  nature  of  the  system  design  and  the 
test  plan  is  systematic  and  thus  a  solid  design  coupled 
with  a  solid  test  plan  will  yield  a  positive  outcome  for 
the  SPARCCS  system  future  development.  The  need  of  the 
customer  defines  the  specific  capability  to  be  fulfilled 
while  the  testing  planning  requirements  determine  the 
degree  of  sophistication  of  the  test  plans  (Kasser,  2007). 

Systems  are  inherently  complex;  the  SPARCCS  system  is 
no  exception.  Through  proper  systems  engineering  test 
design,  that  carefully  decomposes  the  system  into  smaller 
and  smaller  blocks,  measures  of  effectiveness  can  be 
established  so  that  testing  processes  can  be  developed  and 
planned  to  ensure  that  each  level  of  design,  each  building 
block,  performs  correctly  and  that  the  performance  of  that 
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block  can  be  defined  and  measured  through  planned  testing 
(Kossiakoff  and  Sweet,  2011)  .  If  a  system  is  not  tested  in 
a  systematic  manner  that  parallels  the  design  process, 
elements  or  interfaces  may  be  missed  in  the  testing  process 
resulting  in  errors,  faults  or  failures  of  one  or  more  of 
the  design  elements.  In  complex  systems,  a  single  failure 
can  cascade  into  a  larger  problem  resulting  in  total  system 
failure  (Wasson,  2006)  . 

In  wireless  systems,  which  are  inherently  flexible  in 
nature  with  many  simultaneous  users,  errors  that  are  not 
detected  early  in  the  design  phase  can  have  a  significant 
impact  on  the  system  (Viana,  Maag,  &  Zaidi,  2011)  .  The 
current  testing  of  the  SPARCCS  system  was  performed  early 
in  the  design  phase  so  that  future  additions  to  the  system 
would  have  a  solid  set  of  tests  and  validations  on  which  to 
base  new  incremental  designs  of  the  system.  As  such,  the 
overall  final  design  of  the  SPARCCS  system  will  be  more 
robust  with  fewer  errors,  faults,  and  design  flaws. 

3 .  Test  Partitioning 

Systems  testers  must  carefully  decompose  complex 
systems  and  fully  orchestrate  the  testing  of  the  units, 
elements  and  subsystems  to  ensure  that  the  system  is  tested 
properly  and  in  proper  sequence  so  that  interfaces, 
functions,  and  operational  capabilities  are  demonstrated  in 
a  cost  effective  manner  that  leverages  human  and  material 
resources  and  scheduling  (Giadrosich,  1995).  Through  proper 
test  partitioning,  system  integration  is  prevented  from 
becoming  a  testing  and  performance  bottleneck. 

Through  the  proper  test  partitioning  and  sequencing  of 
unit,  element,  and  subsystem  testing,  the  final  stages  of 
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interface,  functionality  and  operational  testing  will  be 
more  viable,  on  schedule,  and  more  complete;  and  the 
verification  of  system  performance  will  be  as  accurate  as 
possible.  In  addition,  through  proper  test  partitioning, 
tests  can  be  grouped  more  accurately  making  testing  more 
cost  and  resource  effective  (Wasson,  2006) . 

Testing  a  complex  system  such  as  the  SPARCCS  system  is 
complex  and  time  and  resource  intensive.  Through  proper 
partitioning,  the  complexity  can  be  managed  and  time  and 
resources  can  be  managed  effectively  and  a  thorough  testing 
of  the  system  can  take  place  to  ensure  proper  operational 
functionality  (Wasson,  2006)  . 

The  SPARCCS  testing  will  have  various  partitions  that 
will  assist  in  the  proper  testing  of  the  system.  These 
partitions  include  various: 

1 .  Operating  systems 

2 .  Internet  browsers 

3.  Hand  held  devices 

4 .  Communications  platforms 

Today' s  wireless  based  systems  are  inherently  and 
exponentially  complex,  especially  systems  that  are  software 
intensive  such  as  the  SPARCCS  system.  If  testing  is  not 
partitioned  properly,  a  system  may  not  be  thoroughly 
tested,  or  if  it  is,  the  testing  may  be  cost  intensive  as 
well  as  time  and  resource  intensive.  Proper  partitioning 
helps  maintain  control  of  the  testing  process  and  helps 
ensure  thorough,  complete,  efficient  and  effective  testing 
(Alena,  et  al . ,  2002). 
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4. 


Controlled  Laboratory  Testing 


A  controlled  test  laboratory  (CTL)  is  critical  to  the 
systems  engineer  for  the  proper  testing  of  a  system.  The 
CTL  is  the  central  entity  that  interconnects  the  elements 
and  subsystems  with  the  tools  that  will  be  used  for  testing 
to  ensure  proper  conduction  of  the  testing  and  to  ensure 
that  the  test  events  are  properly  analyzed  (Ziarco  & 
Krzystan,  2011) . 

CTLs  are  critical  as  they  provide  a  centralized 
location  for  test  functions,  they  provide  an  area  to  stage 
and  practice  test  events  to  ensure  conformity  with  test 
requirements,  and  a  safe  and  controlled  environment  for  the 
tests.  They  ensure  that  tests  are  repeatable.  They  ensure 
that  test  standards  are  followed  through  calibrated  and 
consistent  test  equipment.  They  also  ensure  early  exposure 
to  the  elements  that  will  be  experienced  in  the  operational 
environment  of  the  system  (Sprigg  et  al .  ,  2011)  . 

CTLs  help  complex  systems  to  be  setup  as  decomposed 
subsystems  that  can  be  tested  and  build  back  up  to  full 
system  capability  and  tested  again  in  that  capacity.  CTLs 
thus  provide  a  critical  area  that  supports  testing, 
integration,  integration  testing,  and  qualification  testing 
in  a  controlled,  safe  environment  in  a  centralized  location 
or  locations  (Ziarco  &  Krzystan,  2011)  . 

The  significance  of  a  testing  lab  is  clear:  testing  is 
a  very  complex  process  for  many  systems  and  managing  the 
complex  testing  process  in  a  controlled  manner  for  baseline 
results  is  critical.  CTLs  help  with  this  management. 
Through  a  centralized  location,  that  is  controlled  and 
safe,  system  test  engineers  can  conduct  viable,  efficient 
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tests  that  conform  to  testing  requirements  and  that  help 
simulate  system  environments  and  data  in  a  safe  and 
effective  manner. 

CTLs  help  manage  the  complexity  of  testing  and  help 
organize  and  regulate  the  testing  environment  by  having  all 
of  the  hardware,  software,  simulation  tools,  security,  and 
personnel  available  as  well  as  the  testing  tools, 
instruments,  test  benches,  and  unique  test  hardware  and 
testing  requirements  located  in  specific  physical 

locations.  In  short,  CTLs  help  save  time,  money  and 

resources  and  foster  safety  through  centralized  physical 
locations,  available  instruments  and  tools,  and  help 

provide  for  thorough  testing  through  the  use  of  the 
hardware/software/tools/instruments  that  are  located  at 

each  CTL  (Sprigg  et  al .  ,  2011)  . 

In  the  SPARCCS  testing,  several  controlled  test 
laboratories  were  set  up.  These  SCPACCS  CTLs  provided 
controlled  environments  to  begin  the  necessary  basic 
building  block  testing  of  the  system,  which  comprised  half 
of  the  systems  testing.  Such  CTLs  provided  solid  Internet 
connections,  reliable  power,  protection  from  the  elements, 
and  stable  work  surfaces  on  which  to  work.  They  also 
provided  for  stable  test  areas  where  test  equipment  could 
be  positioned  for  longer  durations  than  with  the  field 
tests . 

In  short,  the  SPARCCS  CTLs  provided  consistent,  stable 
environments  for  initial  systems,  software  and  hardware 
testing,  which  served  as  the  foundation  of  the  test  program 
and  provided  baseline  data  used  in  the  later,  more  complex 
field  tests.  The  CTLs  are  described  in  detail  for  each 
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SPARCCS  test  event  on  each  official  test  form  and  are 
summarized  in  the  test  results  portion  of  this  document. 

5.  Integration  Testing 

In  the  SPARCCS  system  integration  testing  is  critical 
due  to  the  high  software  and  data  sensitive  nature  of  the 
system.  In  many  complex  system-of-systems ,  major  problems 
can  manifest  at  the  interfaces  of  modules  and  systems,  and 
therefore  integration  testing  is  critical  (Sprigg  et  al . , 
2011)  . 

The  SPARCCS  has  two  major  systems  that  need  to  be 
integrated:  the  smart-phone  application  and  the  web-based 
application.  Both  are  complex  systems  in  and  of  themselves. 
Both  also  have  a  wide  array  of  modules  and  systems,  making 
the  integration  of  them  critical  as  well.  By  integrating 
the  systems  and  testing  those  integrations  rigorously 
through  integration  testing,  engineers  can  eliminate  the 
interface  and  interoperability  faults  and  errors,  and  thus 
functional  and  complete  systems  testing  will  go  smoother 
and  be  more  reliable. 

6.  Interface  and  Functional  Testing 

In  the  SPARCCS  system,  interface  verification  should 
take  place  before  functional  testing.  There  are  several 
reasons  for  this.  First,  in  many  software  intensive 
systems,  major  problems  can  manifest  at  the  interfaces  and 
thus  it  is  a  logical  place  to  begin  testing.  Second,  it  is 
easier  to  detect  problems  in  most  cases  at  the  interface 
verification  level.  If  functional  testing  is  done  first, 
excessive  hours  tracing  sources  of  errors  at  the  interfaces 
can  be  wasted  needlessly.  In  short,  interface  verification 
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can  save  time  and  effort  in  the  testing  process.  Interface 
faults  can  disrupt  the  functionality  of  a  system  (Pressman, 
2010) .  Interface  faults  can  include: 


1.  Incorrect  interrupt  handling 

2 .  I/O  Timing 

3.  Call  to  wrong/nonexistent  procedure 

4.  Variable  mismatch 

5.  Parameter  mismatch 

6.  Incompatible  data  types 


As  can  be  seen  in  the  above  list,  major  issues  can 
take  place  at  the  interface  level.  Issues  like  I/O  timing, 
for  example,  could  be  very  difficult  to  diagnose  at  the 
functional  testing  level,  but  could  be  more  easily 
diagnosed  at  the  interface  level.  Procedural  calls,  too, 
are  more  easily  diagnosed  at  the  interface  level.  In  fact, 
almost  all  of  the  faults  listed  above  would  be  more  easily 
diagnosed  at  the  interface  level  (Ziarco  &  Krzystan,  2011) . 

While  it  may  take  more  time  to  get  through  the 
interface  testing  before  getting  to  the  functional  testing, 
it  will  save  time  and  overall  effort  due  to  the  fact  that 
interface  errors  are  common  and  prevalent  in  software 
intensive  systems  (Kossiakoff  &  Sweet,  2011) .  By  nipping 
them  in  the  bud  so  to  speak,  we  make  our  functional  testing 
easier  and  more  reliable. 

7 .  Performance  and  Deployment  Testing 

The  overarching  goal  of  the  SPARCCS  testing  is  to  test 
the  system's  performance  in  the  field.  As  such,  intensive 
performance  and  deployment  testing  will  be  conducted  in 
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various  real  life  scenarios  that  simulate  both  military  and 
police/f ire/EMS  situations. 

Performance  testing  addresses  the  run  time  performance 
of  a  system  within  the  context  of  a  systems  integrated 
platform.  This  type  of  testing  occurs  in  all  phases  of  the 
systems  testing  process  and  requires  both  hardware  and 
software  instrumentation  for  full  effectiveness  (Pressman, 
2010)  .  In  SPARCCS  testing,  performance  testing  will  be  at 
the  forefront  of  all  test  scenarios,  as  the  main  goal  of 
the  current  SPARCCS  testing  is  to  test  the  system  in  its 
current  state  of  development  to  facilitate  improved 
performance  for  the  later  stages  of  design. 

Deployment  testing,  also  known  as  configuration 
testing,  is  structured  to  test  the  system  in  the 

environment  in  which  it  is  designed  to  operate.  Deployment 
testing  tests  the  following  (Pressman,  2010)  : 

1.  Installation  procedures 

2.  Installation  methodologies 

3.  Installation  configurations 

4 .  Operating  systems  platform  conformance 

5.  Internet  browser  platform  conformance 

6.  Hardware  platform  conformance 

In  essence,  deployment  testing  tests  the  actual 
customer  environment  in  which  the  system  will  operate 
including  the  customer' s  installation  of  the  system,  the 

customer' s  configuration  of  the  system,  and  all  other 
issues  related  to  the  actual  real  world  use  of  the  system. 

Deployment  testing  also  covers  the  actual  environments 
in  which  the  system  will  be  used  including  the  physical 
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environments,  the  environmental  settings,  the  potential 
users  and  their  skill  levels,  as  well  as  other  extraneous 
factors  (Pressman,  2010). 

The  SPARCCS  system  testing  will  attempt  to  emulate  the 
various  real  world  uses  of  the  system  including 
installation,  configuration,  and  deployment  of  the  system. 
This  testing  will  encompass  both  the  hardware  and  software 
dimensions  of  the  system  as  well  as  the  specialized  nature 
of  wireless  networking  systems.  In  short,  the  SPARCCS 
testing  will  be  extensive  and  will  begin  with  simple 
installation  tests  and  end  with  complex  real  world  scenario 
testing . 

C .  SOFTWARE  TESTING 

Software  testing  is  the  dynamic  verification  of 
program  behavior  on  a  finite  set  of  test  cases  that  are 
carefully  and  suitably  selected  from  what  is  usually  an 
infinite  domain  and  are  run  against  the  expected  behavior 
of  the  system  (Bertolino,  2001)  .  Software  testing  should  be 
geared  towards  problem  prevention,  and  all  quality  testing 
should  point  to  the  avoidance  of  problems  in  the  final 
system  (IEEE  SWEBOK,  2004) .  Software  testing  is  a  mandatory 
part  of  testing  and  system  development,  as  it  evaluates  the 
quality  of  the  software  product  in  the  system  by 
identifying  performance  defects  and  problems  and  directly 
helps  improve  the  system  (Bertolino,  2001) . 
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Most  wireless  ad-hoc  systems  today  are  software 
intensive  (Viana,  Maag,  &  Zaidi,  2011)  .  The  SPARCCS  system 
is  no  exception.  Small  software  errors  can  cascade  into 
major  errors  or  faults  that  can  have  catastrophic 
consequences  (Jorgensen,  2002). 

D .  HARDWARE  TESTING 

The  hardware  in  the  SPARCCS  system  is  critical  to  the 
system  success.  In  the  system,  several  pieces  of  hardware 
are  utilized: 

1 .  Laptop  computers 

2 .  Smart  phones 

3.  Handheld  tablet  PCs  (7"  and  10") 

4 .  Various  Antennae 

5.  WIFI  Jet  Pack 

6.  Satellite  communication  systems  (BGAN) 

7 .  Wave  Relay  and  WIMAX  devices 

Each  piece  of  hardware  must  thus  be  partitioned  and 
tested  before  being  integrated  into  the  system  for  systems 
testing  (Wasson,  2006,  &  Alena,  et  al .  ,  2002)  .  Hardware 

tests  will  include  basic  functionality,  internal  systems 
functionality,  and  designed  functionality  and 

communications  capability  in  the  wireless  spectrum. 

All  SPARCCS  hardware  platforms  will  be  tested  in  the 
beginning  of  the  test  sequences  to  ensure  that  the  SPARCCS 
software  functionality  is  its  inherent  functionality  and 
not  a  result  of  hardware  issues.  Proper  documentation  of 
the  hardware  testing  will  be  conducted  to  ensure  proper 
reference  during  the  overall  systems  testing.  Hardware 


testing  will  be  initially  conducted  in  the  indoor  and 
outdoor  laboratories,  which  are  described  later  in  this 
document . 

Additionally,  the  SPARCCS  hardware  will  be  tested  in 
various  combinations  to  ensure  that  there  are  no 
compatibility  issues  between  the  hardware  platforms.  This 
testing  will  begin  with  basic  configurations  and  advance  to 
more  complex  configurations  to  ensure  full  systems 
compatibility  among  the  various  hardware  devices. 

E.  WHEN  TO  STOP  TESTING 

The  process  of  testing  in  a  system  can  be  literally 
endless.  The  test  program  cannot  be  continued  until  all  of 
the  faults,  errors  and  defects  are  found,  as  this  is 
economically  impossible  and  infeasible  in  terms  of  time, 
money,  and  resources.  As  such,  the  scope  and  limits  of  the 
testing  must  be  firmly  established.  Testing  is,  in  essence, 
a  tradeoff  between  time,  budget,  and  system  guality  (Pan, 
2012)  . 

The  SPARCCS  testing  will  encompass  a  selected  set  of 
test  scenarios  and  a  limited  number  of  hardware  platforms. 
In  addition,  the  wireless  communications  technologies  will 
be  limited  to  those  that  would  realistically  be  feasible  in 
the  real  operational  environment. 

Overall,  the  goal  of  SPARCCS  testing  is  to  encompass 
enough  test  scenarios  to  establish  a  solid  demonstration  of 
the  system  performance  at  its  current  state  of  development. 
While  limited  in  scope,  the  testing  will  provide  guidance 
to  the  current  SPARCCS  developers  and  will  help  them  avoid 
negative  issues  as  the  development  continues. 
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F .  CHAPTER  SUMMARY 

The  testing  plan  for  any  system  is  critical  for  the 
long-term  success  of  the  project.  The  earlier  a  problem  is 
detected  in  the  system' s  development  the  less  expensive  it 
is  to  fix  and  the  less  time  the  system  will  be  in  the 
repair  stage  for  that  issue  (Giadrosich,  1995,  Sprigg  et 


al . , 

2011,  &  Biemer,  2010)  . 

The 

systems 

engineering  test  methodology 

was  chosen 

for 

the 

SPARCCS 

system  since  SPARRCS  is  a  complex  system 

with 

hardware 

components,  software 

components , 

communication  protocols  and  methodologies  and  several 
systems  interfaces.  Through  thorough  and  progressive 
systems  testing,  which  is  partitioned  properly  and 
constructed  properly,  the  SPARCCS  system  will  have  a  solid 
testing  foundation  to  assist  in  the  elimination  of  systems 
errors,  faults,  design  flaws  and  other  technical  issues, 
facilitating  a  more  expeditious  and  technically-sound 
development  of  the  system  in  its  later  stages. 

The  next  chapter  will  discuss  the  requirements  of  the 
system  as  they  relate  to  the  testing  of  the  system.  The 
test  requirements  will  be  developed  and  matched  to  specific 
testing  levels.  The  phases  of  testing  will  be  discussed  as 
will  the  qualification  and  audit  plans  for  SPARCCS  testing. 
These  plans  will  help  with  the  quality  assurance  of  the 
testing  and  adherence  to  proper  testing  procedures. 
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Ill .  SPARCCS  TECHNOLOGY  INTEGRATED  TEST  PLAN 


This  chapter  develops  an  integrated  test  plan  for 
SPARCCS.  It  begins  with  a  thorough  requirements  analysis 
for  the  specific  scope  of  the  system  that  will  be  tested  in 
this  thesis.  The  requirements  are  placed  in  a  Verification 
Cross  Reference  Matrix  (VCRM) ,  which  is  a  requirements 
engineering  tool  for  managing  system  requirements.  The  test 
plan  approach  is  then  developed  to  demonstrate  the 
comprehensive  performance  testing  approach,  the  levels  of 
testing  and  the  various  test  environments.  The  chapter 
concludes  with  the  qualification  and  audit  plans  to  ensure 
that  the  test  plan  and  sequences  are  accurate  and 
appropriate  for  the  levels  and  scope  of  the  required 
SPARCCS  testing. 

A.  INTRODUCTION 

The  customers  of  SPARCCS  are  military,  police,  fire, 
and  EMS  at  the  local,  regional  and  federal  levels.  As  such, 
a  set  of  requirements  must  be  derived  from  the  projected 
use  of  the  system.  A  CONOPS  (Concept  of  Operations)  for 
SPARCCS  has  been  developed  as  presented  in  Appendix  A. 

Our  test  plan  initiates  from  an  analysis  of  the  system 
requirements.  The  following  section  will  develop  the 
SPARCCS  requirements  that  will  serve  as  the  foundation  for 
the  test  events  in  the  SPARCCS  testing  plan. 

B.  TEST  REQUIREMENTS  ANALYSIS 

1 .  Requirements  Analysis  Description 

In  the  SPARCCS  system,  there  are  57  individual  test 
requirements  statements  derived  for  the  system.  These 
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requirements  were  derived  from  one  of  three  dimensions  of 
the  system:  the  entire  SPARCCS  system,  the  smart  phone 
application,  and  the  web  application. 

Each  requirement  was  analyzed,  evaluated  and  placed 
into  the  VCRM  in  either  an  unchanqed  state  or  in  a  modified 
or  split  state,  depending  on  the  level  of  precision  needed 
for  each  particular  requirement,  as  derived  from  the 
current  system  documentation.  Careful  consideration  was 
given  to  the  clarity,  specificity,  and  testability  of  each 
requirement  placed  in  the  VCRM  to  ensure  precision  in  the 
test  cases  derived  from  each  specific  requirement. 


Requirements  Testing  Distribution 


5% 


■  Demonstration 


■  Test 


■  Inspection 

■  Analysis 


Figure  4.  Requirements  Test  Distribution 
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2. 


VCRM  Matrix 


Our  VCRM  as  shown  in  Table  1  contains  57  overall 
system  test  requirements  that  were  derived  to  serve  as  the 
foundation  of  the  SPARCCS  testing  program.  The  matrix 
consists  of  a  unique  requirements  identification  number  for 
each  test  item,  a  description  of  the  actual  requirement, 
the  method  of  testing,  and  the  level  of  testing.  The 
methods  of  testing  a  system  are:  demonstration  (D)  ,  test 
(T) ,  inspection  (I),  and  analysis  (A).  The  levels  of 
testing  are:  hardware,  software,  and  systems 


Table  1 .  SPARCCS  VCRM  Matrix 


Req  ID 

Requirement 

Method 

Level 

001 

The  SPARCCS  system  shall  operate  on  Android 

smart  phone  hardware. 

D 

Hardware 

002 

The  SPARCCS  system  shall  operate  on  Android 

tablet  PCs  of  various  sizes. 

D 

Hardware 

003 

The  SPARCCS  system  shall  operate  on  the 

Android  operating  system  of  version  2.2  or 

greater. 

T 

Software 

004 

The  SPARCCS  command  system  shall  operate  on 

any  modern  web  browser. 

D 

Software 

005 

The  SPARCCS  system  shall  operate  on  the 

Google  cloud  application  platform  (Google  App 

Engine). 

T 

Systems 

006 

The  SPARCCS  system  shall  operate  on  and  fully 

leverage  the  Google  Datastore  platform. 

T 

Software 

007 

The  SPARCCS  system  shall  operate  on  [Gi]  the 

Android  SQLite  database  platform. 

T 

Software 

008 

The  SPARCCS  system  shall  operate  under 

HTTPPosts  and  HTTPRequests  using  HTTP 

servlets  for  Android  to  server  communications. 

T 

Software 
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Req  ID 

Requirement 

Method 

Level 

009 

The  SPARCCS  system  shall  operate  under 

Remote  Procedure  Calls  for  web  client  to  server 

communications. 

T 

Software 

010 

The  SPARCCS  system  shall  utilize  a  user  login 

system. 

1 

Software 

011 

The  SPARCCS  system  shall  utilize  the 

commercially  available  GPS  system  present  on 

the  user  device. 

1 

System 

012 

The  SPARCCS  system  shall  utilize  Google  Maps 

as  the  visual  map  interface. 

1 

Software 

013 

The  user  login  system  shall  enable  user  accounts 

to  be  created  on  the  Android  application  or  on  the 

command  program. 

D 

Software 

014 

The  user  login  system  shall  acquire  user  first 

name,  middle  initial,  and  last  name;  email 

address,  phone  number,  unit,  and  password. 

D 

Software 

015 

The  user  login  system  shall  present  5  types  for 

user  login  registration:  military,  fire,  medical, 

humanitarian  assistance,  or  law  enforcement. 

D 

Software 

016 

The  user  login  system  shall  create  a  username 

with  a  unique  and  meaningful  user  ID. 

D 

Software 

017 

The  user  interface  shall  allow  a  user  to  create  a 

mission. 

D 

Software 

018 

The  user  interface  shall  allow  a  user  to  join  a 

mission. 

T 

Software 

019 

The  user  interface  shall  allow  a  user  to  edit  a 

mission. 

T 

Software 

020 

The  user  interface  shall  allow  a  user  to  view  a 

mission. 

D 

Software 

021 

The  user  interface  shall  allow  a  user  to  create  a 

point  of  interest. 

T 

Software 

022 

The  user  interface  shall  allow  a  user  to  join  a  point 

of  interest. 

T 

Software 
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Req  ID 

Requirement 

Method 

Level 

023 

The  user  interface  shall  allow  a  user  to  edit  a  point 

of  interest. 

T 

Software 

024 

The  user  interface  shall  allow  a  user  to  view  a 

point  of  interest. 

D 

Software 

025 

The  application  shall  have  the  ability  to  capture 

images  and  place  them  into  the  system. 

T 

System 

026 

The  application  shall  have  the  ability  to  edit 

images  from  a  mission. 

T 

Software 

027 

The  application  shall  have  the  ability  to  view 

images  from  a  mission. 

T 

Software 

028 

The  application  shall  have  the  ability  to  delete 

images  from  a  mission. 

T 

Software 

029 

The  application  shall  have  the  ability  to  view  all 

missions  on  a  Google  map. 

T 

Software 

030 

The  application  shall  have  the  ability  to  view  all 

points  of  interest  on  a  Google  map. 

T 

Software 

031 

The  application  shall  have  the  ability  to  view  all 

images  on  a  Google  map. 

T 

Software 

032 

The  application  shall  have  the  ability  to  view  all 

missions  in  a  list  format. 

T 

Software 

033 

The  application  shall  have  the  ability  to  view  all 

points  of  interest  in  a  list  format. 

T 

Software 

034 

The  application  shall  have  the  ability  to  view  all 

images  in  a  list  format. 

T 

Software 

035 

The  application  shall  have  the  ability  to  view  all 

responders  in  a  list  format. 

T 

Software 

036 

The  application  shall  have  the  ability  to  retrieve 

GPS  location  from  the  host  hardware. 

T 

Systems 

037 

The  application  shall  have  the  ability  to  correlate 

GPS  location  with  missions,  points  of  interest,  and 

images. 

T 

Systems 

038 

The  application  shall  have  the  ability  to  store 

mission,  image,  and  point  of  interest  data  on  the 

SQLite  database. 

T 

Systems 
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Req  ID 

Requirement 

Method 

Level 

039 

The  application  shall  have  the  ability  to  store 

mission,  image,  and  point  of  interest  data  on  the 

Google  Datastore  database. 

T 

Systems 

040 

The  application  shall  have  the  ability  to  sync  data 

between  the  SQLite  database  and  the  Google 

Datastore  database. 

T 

Systems 

041 

The  application  shall  collect  information 

concerning  the  mission  including  userlD,  mission 

name,  mission  description,  mission  leader, 

mission  creator,  and  mission  information. 

D 

Software 

042 

The  application  shall  collect  information  on  a  point 

of  interest  including  userlD,  description,  time, 

creator,  correlating  mission,  location  notes,  and 

latitude  and  longitude. 

D 

Software 

043 

The  application  shall  place  a  gold  flag  on  the 

Google  map  of  the  point  of  interest’s  creator  at  the 

location  of  creation  within  2  meters. 

D 

Software 

044 

The  application  shall  place  a  gold  flag  on  the 

Google  map  of  the  point  of  interest  of  all  members 

of  the  mission. 

D 

Systems 

045 

The  application  shall  collect  information  on  an 

image  including  userlD,  description,  time,  creator, 

correlating  mission,  location  notes,  and  latitude 

and  longitude. 

D 

Software 

046 

The  application  shall  place  a  camera  icon  on  the 

Google  map  of  the  image  creator  at  the  location  of 

creation  within  2  meters. 

D 

Software 

047 

The  application  shall  place  a  camera  icon  on  the 

Google  map  at  the  location  of  the  photograph. 

D 

Systems 

048 

The  headquarters  application  shall  have  a  list 

menu  listing  all  missions,  points  of  interest, 

responders,  photos,  map  options,  and  a  live 

Google  map  of  the  mission  area. 

D 

Software 
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Req  ID 

Requirement 

Method 

Level 

049 

The  headquarters  application  shall  have  the  ability 

to  click  on  an  icon  on  the  Google  Map  and  have 

the  mission  details  listed. 

D 

Software 

050 

The  headquarters  application  shall  have  the  ability 

to  display  accurate  mission  statistics  in  real  time. 

A 

Software 

051 

The  headquarters  application  shall  have  the  ability 

to  display  accurate  point  of  interest  statistics  in 

real  time. 

A 

Software 

052 

The  headquarters  application  shall  have  the  ability 

to  display  accurate  responder  statistics  in  real 

time. 

A 

Software 

053 

The  headquarters  application  shall  have  the  ability 

to  display  accurate  photos  on  the  photo  panel  in 

real  time. 

D 

Software 

054 

The  SPARCCS  system  shall  operate  on  a 

commercial  WIFI  network. 

T 

Systems 

055 

The  SPARCCS  system  shall  operate  on  a  BGAN 

satellite  network. 

T 

Systems 

056 

The  SPARCCS  system  shall  operate  on  a  hybrid 

BGAN/Wave  Relay  network. 

T 

Systems 

067 

The  SPARCCS  system  shall  operate  on  a  hybrid 

BGAN/WIMAX  network. 

T 

Systems 

C.  TEST  APPROACH,  FLOW,  AND  SEQUENCE 
1 .  Overall  Approach 

Live  systems  testing  will  be  the  primary  quantitative 


method  of 

research  used  in 

this  study.  All 

data 

will 

be 

extracted 

from  live 

test 

cases  and  all 

data 

will 

be 

original 

to  this 

research.  Proper  statistical 

and 

quantitative  analysis 

will 

be  conducted  on 

the 

data 

to 

formulate  conclusions  about  the  system.  While  qualitative 


33 


methods  will  be  used  in  some  tests,  their  results  will  be 
quantized  and  analyzed  as  quantitative  data. 

The  test  approach  will  involve  four  discrete  areas: 

1 .  The  development  of  test  requirements 

2 .  The  three-phase  test  plan 

3.  The  qualification  plan 

4 .  The  audit  plan 

The  SPARCCS  test  approach  will  fulfill  these  areas 
sequentially  to  ensure  complete  testing,  review  and 
auditing.  The  sequential  nature  of  the  test  approach  will 
ensure  that  all  test  items  are  carefully  set  up  with  pre¬ 
conditions  and  carefully  reviewed  and  audited  to  ensure  the 
clarity,  accuracy,  precision,  and  complete  fulfillment  of 
the  SPARCCS  test  requirements. 

The  testing  plan  has  several  overall  divisions.  First, 
a  comprehensive  set  of  test  requirements  will  be  developed 
to  ensure  the  formality  of  the  testing  program  and  to 
ensure  the  comprehensive  nature  and  strict  organization  of 
the  test  program.  Prerequisite  requirements  conditions  will 
be  analyzed  and  set  up  to  ensure  that  all  of  the  conditions 
are  in  place  before  testing  will  begin.  The  test  engineer 
will  ensure  and  verify  that  these  conditions  are  fulfilled 
before  the  commencement  of  formal  testing. 

Second,  the  testing  will  begin.  The  SPARCCS  tests  are 
categorized  into  three  phases  with  gradual  progression  of 
difficulty  and  architectural  system  involvement.  Through 
the  systematic  testing  of  the  four  phases,  a  complete  set 
of  tests  will  be  performed  covering  start  up,  system 
operations,  system  performance,  and  abnormal  conditions. 
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Third,  a  comprehensive  field-testing  program  will  be 
established  and  conducted  in  a  live  scenario  to  ensure  the 
real  world  operability  of  the  SPARCCS  system. 

Finally,  once  SPARCCS  testing  is  complete,  the  review 
and  audit  plan  will  be  conducted  to  review  the  issues, 
modifications,  and  agreement  regarding  each  test  item. 
Audits  are  conducted  for  each  test  to  ensure  that  the 
procedures  are  followed  correctly  and  test  results  are 
valid  and  accurate. 

2 .  Phases  of  Testing 

SPARCCS  tests  will  be  categorized  into  three  phases 
with  gradual  progression  of  difficulty  and  system 
involvement.  These  phases  are  depicted  in  Table  2. 


Table  2 .  Phases  of  SPARCCS  Testing 


Phase 

Description 

1 

Perform  start-up  testing:  The  purpose  of  this  test 

phase  is  to  assess  the  initial  system  capability  of 

start-up,  log-in,  and  initialization. 

2 

Perform  operation  related  testing:  The  purpose  of 

this  test  phase  is  to  assess  system  performance 

under  normal  operation.  The  tests  will  be  conducted 

in  the  test  labs,  both  indoor  and  outdoor,  under 

controlled  conditions. 

3 

Field  tests:  the  system  will  be  subjected  to  live 

field-testing  in  full  operating  conditions  by  a 

team  of  testers. 
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The  field  tests  will  be  conducted  in  the  tactical 
areas  of  the  former  Ft.  Ord,  a  decommissioned  US  Army 
installation  that  was  the  home  to  a  full  infantry  division. 
The  remote  areas  of  Ft.  Ord  provide  tactical  simulation 
environment  similar  to  what  actual  users  of  the  SPARCCS 
system  will  experience.  The  region  has  a  plethora  of 
terrains,  such  as  heavily  wooded  areas,  flatlands,  fields, 
extreme  hills  and  valleys,  as  well  as  mixed  terrains. 

D.  QUALIFICATION  PLAN  FOR  ACCURATE  TESTING 

The  development  of  a  testing  plan  must  include 
qualification  conditions  that  ensure  the  accuracy  and 
integrity  of  the  tests  and  the  overall  plan  (Ziarco  & 
Krzystan,  2011,  Sprigg  et  al .  ,  2011)  .  Table  3  details  the 

qualification  conditions  of  the  SPARCCS  testing  that  ensure 
proper  and  accurate  testing  of  the  system.  The  conditions 
will  be  reviewed  for  each  test  event  and  an  analysis  and 
reconciliation  of  the  conditions  will  occur  with  each  test 
event  and  will  be  documented  with  the  event. 


Table  3.  SPARCCS  Test  Qualification  Conditions 


Condition 

Description 

1 

Before  tests  commence  for  the  SPARCCS 

system,  there  needs  to  be  assurance  that 

the  system  has  been  completely  built  up  to 

the  required  platform  architecture  for 

testing  and  integrated  and  that  development 

for  the  current  build  has  concluded. 

Existing  issues  and  outstanding  concerns 

from  the  development  phase  needs  to  be 
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Condition 

Description 

resolved  before  testing  can  begin. 

2 

The  SPARCCS  software  and  system  components 

such  as  phones,  laptops,  servers,  routers, 

etc.  are  under  formal  configuration 

management  to  ensure  that  changes  during 

the  testing  phase  will  be  truthfully 

reflected  and  recorded.  During  actual 

testing,  target  hardware  and  software  that 

reflect  actual  operation  are  used  and  need 

to  be  ready  before  testing. 

3 

In  order  to  have  test  integrity,  test 

planning  must  be  conducted  in  the  form  of  a 

meeting  to  brief  SPARCCS  test  members  and 

discuss  testing  detail.  Some  tasks  under 

test  planning  include  actions  to  refine 

developed  test  procedures  and  confirm  the 

tools  and  resources  needed  for 

qualification-testing  are  ready  for  use. 

4 

To  ensure  the  quality  of  individual  SPARCCS 

tests,  there  must  be  multiple  testing  dry 

runs  to  refine  test  procedures,  correct 

existing  issues,  and  confirm  system 

performance . 

5 

Performance  of  test  readiness  review  must 

be  used  to  determine  system  readiness, 

evaluate  results  of  earlier  dry  runs, 

define  roles  and  responsibilities  of  test 
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Condition 

Description 

team,  report  validation  of  test  tools, 

provide  test  schedule,  define  retest 

criteria,  and  define  success  criteria. 

E.  CONTROLLED  TEST  CASES 

The  purpose  of  controlled  test  cases  is  to  test  the 
SPARCCS  system  in  a  structured  environment  with  limited 
variables  and  set  conditions  to  gain  an  initial  baseline 
for  the  system  without  undue  influences  from  extraneous 
external  variables  (Giadrosich,  1995;  Ziarco,  &  Krzystan, 
2011) .  An  initial  controlled  environment  is  critical  for 
the  SPARCCS  system  to  ensure  that  it  has  basic 
functionality,  that  the  interfaces  are  functional,  and  that 
the  overall  system  is  functioning  as  designed. 

Since  this  research  is  the  initial  set  of  tests  for 
the  SPARCCS  system,  an  extensive  set  of  controlled  tests  is 
vital  to  ensure  proper  continued  development  of  the 
software  and  to  ensure  that  the  field  tests  are  conducted 
with  a  solid  system  foundational  base. 

The  controlled  tests  of  the  SPARCCS  system  shall  be 
conducted  in  two  basic  environments,  an  indoor  lab 
environment  and  a  limited,  confined  outdoor  environment. 
These  environments  will  be  the  same  throughout  controlled 
testing  to  ensure  consistency  and  to  ensure  the  development 
of  accurate  and  systematic  system  baselines. 
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F.  FIELD  TEST  PLAN 

1 .  Purpose 

Field-testing  (also  known  as  live  environmental 
testing)  assures  that  the  system  will  perform  in  its 
expected  environments  as  required,  when  required,  and  as 
long  as  required.  A  system  can  be  tested  to  ensure  that  it 
meets  specifications,  but  if  the  system  does  not  operate 
well  in  its  target  environment,  the  fact  that  it  meets 
specifications  may  be  a  moot  point. 

SPARCCS'  field-testing  will  be  conducted  at  the  former 
Ft.  Ord  to  ensure  that  the  SPARCCS  system  performs  well  in 
the  actual  operational  environment.  Tests  will  cover  all 
areas  not  fully  tested  in  the  wireless  lab  and  controlled 
test  areas.  The  SPARCCS  field  tests  will  be  designed  to 
improve  system  confidence  and  to  thoroughly  cover  issues 
that  may  arise  in  the  real  world.  The  tests  will  also  be 
designed  to  uncover  areas  that  may  have  been  missed  in  lab 
and  controlled  testing. 

2 .  Field  Test  Descriptions 

The  test  descriptions  for  the  field  tests  are 
contained  in  Table  4.  Each  test  will  have  its  own  set  of 
test  sheets  for  formal  test  documentation  while  the  test  is 
being  conducted.  Each  test  will  have  a  specific  test 
location  listed  in  the  table,  with  further  description  of 
the  location  following  the  table.  The  table  will  list  the 
specific  requirement  or  requirements  with  which  the  test 
corresponds . 

As  with  the  controlled  testing  scenarios,  all  test 


cases  must  correspond  with  at  least  one  test  requirement 

for  proper  systems  testing  (Sprigg,  Krzystan,  &  Ziarco, 
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2010) .  Additionally,  the  level  of  confidence  will  be 
annotated  on  the  table  for  each  test  case. 


Table  4 .  Field  Test  Descriptions 


Test 

Test 

Test  Description 

Limitations/ 

Number 

Location 

Constraints 

FT1 

Camp 

General  system  testing 

Limited  on 

ability 

Roberts 

in  the  field,  multiple 

to  maneuver 

vehicle 

user  testing,  live 

on  Army 

range . 

system  tracking 

Limited  number  of 

testing,  WIFI  testing. 

users . 

Time 

Mission  creation  and 

limitation 

to 

POI  creation. 

daylight 

hours . 

photograph  and  mission 

Possible  limitation 

note  creation,  extended 

on  wireless 

service 

use  testing. 

based  on  location 

of  testing  in 

relation  to 

nearest 

service  tower. 

FT2 

Camp 

Extensive  testing  of 

Time  limitation  to 

Roberts 

Wave  Relay  and  WIMAX 

daylight 

hours . 

systems  over  8  miles  of 

Stationary 

terrain,  compare  and 

positioning 

of 

contrast  Wave  Relay  and 

wireless  devices. 

WIMAX  performance,  BGAN 

Limitations 

on 

Internet  connectivity. 

device  positioning 

SPARCCS  system  usage 

due  to 

power 

over  WIMAX  and  Wave 

requirements 

(120 

Relay  with  BGAN 

v)  of  Wave 

Relay 

connectivity.  Mission 

and  WIMAX  devices. 
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Test 

Number 

Test 

Location 

Test  Description 

Limitations/ 

Constraints 

creation,  POI  creation, 

precision  tracking, 

photographic 

capabilities,  mission 

note  capabilities, 

overall  SPARCCS 

functionality . 

requires  power 

source  and  elevated 

location  as  well  as 

line  of  sight 

requirement  for 

WIMAX  network. 

FT3 

MIRA 

Marina 

Campus 

Extensive  testing  of 

SPARCCS  in  a  multi¬ 
building  campus  type 

setting.  Buildings  of 

multiple  types  of 

construction  and 

dimensions/layouts . 

WIFI  and  BGAN  testing. 

Mission  creation,  POI 

creation,  photographic 

capabilities,  mission 

note  capabilities,  GPS 

positioning  overall 

SPARCCS  functionality. 

Possible  limitation 

on  WIFI 

connectivity  due  to 

proximity  to  tower. 

FT4 

MIRA 

Chews 

Ridge 

Location 

High  altitude,  remote 

setting  wilderness 

testing.  BGAN  testing. 

GPS,  Mission  creation, 

POI  creation, 

photographic 

capabilities,  mission 

BGAN  testing  only, 

no  WIFI  signal 

available . 

Difficult  location 

to  travel  to. 

Remote  setting. 
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Test 

Number 

Test 

Location 

Test  Description 

Limitations/ 

Constraints 

note  capabilities,  GPS 

positioning  overall 

SPARCCS  functionality. 

FT5 

Former 

Ft  Ord 

Concrete 

Test 

Range 

Precision  testing  on 

WIFI,  and  BGAN  on 

level,  concrete  test 

range.  Distance  signal 

testing.  SPARCCS  system 

testing  and 

functionality  testing. 

Possible  limitation 

on  WIFI 

connectivity  due  to 

proximity  to  tower. 

FT  6 

Former 

Ft  Ord 

Wildland 

Testing 

Testing  SPARCCS  with 

WIFI  and  BGAN 

connectivity  in  various 

states  of  wildland 

configuration:  low 

shrubs  to  thick  woods. 

SPARCCS  system  testing 

and  functionality 

testing . 

Possible  limitation 

on  WIFI 

connectivity  due  to 

proximity  to  tower. 

Possible  issues 

with  line  of  sight 

BGAN  issues  with 

satellite  due  to 

woods . 

FT7 

Monterey 

County, 

CA 

Highways 

Precision  tracking 

using  WIFI.  Travel  100 

miles  in  one  direction 

with  various  stops. 

Mission  creation,  POI 

creation,  GPS 

positioning  precision, 

movement  tracking, 

mission  notes. 

Possible  breaks  in 

WIFI  connectivity 

due  to  location  of 

traveling  vehicle. 
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Test 

Number 

Test 

Location 

Test  Description 

Limitations/ 

Constraints 

photographic  imaging, 

multi  user  tracking. 

G.  TEST  EQUIPMENT 

The  testing  of  systems  software  requires  various 
pieces  of  test  equipment,  and  the  SPARCCS  testing  is  no 
different.  The  equipment  can  be  broken  down  into  three 
categories:  test  support  equipment,  test  platforms,  and 
wireless  technologies.  The  following  sections  briefly 
describe  the  equipment. 

1 .  Test  Support  Equipment 

The  test  support  equipment  used  in  the  SPARCCS  testing 
was  vital  for  system  positioning  and  system  safety.  Figure 
5  depicts  the  test  support  equipment  used  in  the  SPARCCS 
testing . 


Figure  5.  Test  Support  Equipment 
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Equipment  includes  two  equipment  stands  with  ridqes 
for  laptop  and  communication  device  support,  a  rubber  mat 
for  device  support  and  high  visibility  cones  for  safety 
purposes.  A  precision  land-measuring  wheel  is  included  in 
the  equipment  for  measurement  of  land  to  the  nearest  foot. 
A  compass  is  included  to  assist  in  the  pointing  of  the 
BGAN,  Wave  Relay,  and  WIMAX  devices. 

2 .  Wireless  Technology  Equipment 

Wireless  technology  is  at  the  heart  of  the  SPARCCS 
testing.  The  technologies  that  will  be  tested  are  the  BGAN 
system.  Figure  6,  the  Verizon  WIFI  4G  LTE  Jet  Pack,  Figure 
7,  the  Wave  Relay  System,  Figure  8,  and  the  WIMAX  system. 
Figure  9.  The  equipment  will  be  described  in  detail  in  the 
testing  section  as  well  as  in  the  appendices. 


Figure  6.  Hughes  9202  Inmarsat  BGAN  System 
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Figure  7.  Verizon  4G  LTE  Jetpack 


Figure  8.  Persistent  Systems  Wave  Relay  Radio 
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3 .  Wireless  Test  Devices 


The  SPARCCS  system  is  designed  to 
devices.  As  such,  various  Android  devices 
tests  of  this  research.  Table  5  lists  the 
the  testing  of  the  SPARCCS  system. 


work  on  Android 
are  used  in  the 
devices  used  in 
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Table  5.  Test  Devices 


Device  Type 

Make 

Model 

Android 

Version 

Smartphone 

HTC 

EVO 

2.2 

Smartphone 

Motorola 

Droid  X 

2.3.4 

Smartphone 

LG 

Enlighten 

2.3 

Tablet  (7") 

Samsung 

Galaxy 

4.0.3 

Tablet  (7") 

Asus 

A100 

3.2.1 

Tablet  (10") 

Toshiba 

AT300 

4.0.3 

H .  CHAPTER  SUMMARY 

The  proper  setup  of  a  testing  plan  is  critical  for  the 
overall  success  of  the  testing.  This  chapter  developed  the 
system  requirements  to  facilitate  the  accurate  development 
of  the  test  cases  ensuring  that  the  tests  are  completed 
correctly  and  ensuring  that  the  correct  tests  are 
completed.  The  qualification  plan  was  developed  to  ensure 
that  the  system  is  tested  with  a  high  level  of  quality 
assurance.  The  controlled  test  plans  and  field  test  plans 
were  developed  to  demonstrate  the  depth  of  the  testing 
program.  Finally,  the  equipment  used  in  the  testing  was 
introduced . 
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IV.  TEST  RESULTS 


This  chapter  covers  all  of  the  SPARCCS  tests  in 
detail.  It  begins  with  the  controlled  tests  and  the  results 
of  those  tests,  followed  by  the  field  tests.  An  emphasis  on 
graphics  and  images  can  be  found  in  this  chapter  to 
demonstrate  the  specific  test  environments,  test 
conditions,  and  test  results. 

A.  CONTROLLED  TESTING 

Controlled  testing  occurred  in  several  locations. 
Indoor  testing  was  conducted  in  an  indoor  residential 
testing  laboratory.  Outdoor  controlled  testing  occurred  at 
several  locations  including  a  small  wooded  area,  the  campus 
of  the  California  Department  of  Forestry  and  Fire 
Protection  in  Monterey,  California,  and  at  the  Naval 
Postgraduate  School.  Appendix  A  contains  the  field  notes 
from  the  controlled  testing. 

1.  Handheld  Device  Testing 

The  wireless  devices  used  in  the  field-testing  were 
tested  rigorously  over  the  course  of  several  weeks  to 
ensure  their  viability.  As  new  devices  were  acquired,  they 
were  put  through  the  same  set  of  tests  to  ensure  field  test 
consistency  and  accuracy.  Table  6  describes  the  handheld 
test  details. 


49 


Table  6. 


Handheld  Test  Details 


Test  Dates 

May  3-26,  2012,  June  11-18,  2012 

Test  Personnel 

Donna  Dulo 

Equipment 

All  listed  handheld  devices  of  Table 

5,  Verizon  Jetpack  with  Antenna 

Test  Software 

GPS-Test,  SpeedTest 

Test  Locations 

Indoor  Test  Lab  in  Seaside,  CA 

Test  Objectives 

To  determine  the  viability  of  the 

handheld  devices  to  be  used  in  the 

field  tests. 

The  handheld  testing  was  comprised  of  several 
components.  The  first  was  GPS  testing  through  a  program 
called  GPS-Test  that  tested  the  functionality  of  GPS  on  the 
device.  The  second  test  was  a  basic  wireless  connectivity 
test  where  each  device  was  connected  to  the  Verizon  Jetpack 
network  to  test  its  functionality  on  a  network.  The 
SpeedTest  software.  Figure  10,  was  used  to  see  that  data 
was  being  uploaded  and  downloaded  to  the  device.  The 
SpeedTest  software  was  configured  in  "high  test" 
effectiveness  mode,  as  it  was  in  all  tests  in  this  research 
to  ensure  test  accuracy.  This  mode  decreased  the  graphics 
of  the  software  in  lieu  of  more  accurate  data  flow 
readings . 

The  third  test  was  to  test  the  ability  of  SPARCCS  to 
be  loaded  properly  on  the  device.  This  test  loaded  SPARCCS 
from  the  Gmail  system  directly  on  each  device.  The  final 
test  was  the  operating  system  test,  which  tested  the 
ability  of  SPARCCS  to  open  as  an  application  on  the 

device's  specific  operating  system. 
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Figure  10.  Speed  Test  Testing  Software 

The  results  of  the  testing  are  found  in  Table  7 .  Note 
the  specific  operating  system  issue  with  the  HTC  smart 
phone  device  that  contained  the  Android  Operating  System 
(OS)  version  2.2. 
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Table  7.  Results  of  Device  Basic  Functionality  Testing 


Device 

OS 

GPS 

Test 

Wireless 

Connectivity 

Test 

SPARCCS 

Application 

Install 

Test 

SPARCCS 

Basic 

Operation 

Test 

HTC  EVO 

2.2 

Passed 

Passed 

Passed 

Failed 

Droid  X 

2.3.4 

Passed 

Passed 

Passed 

Passed 

Enlighten 

2.3 

Passed 

Passed 

Passed 

Passed 

Galaxy 

4.0.3 

Passed 

Passed 

Passed 

Passed 

A100 

3.2.1 

Passed 

Passed 

Passed 

Passed 

AT300 

4.0.3 

Passed 

Passed 

Passed 

Passed 

The  HTC  smart  phone  was  not  able  to  run  the  SPARCCS 
application  so  further  testing  into  this  issue  was 
warranted.  A  second  phone  of  equal  make,  model,  and  OS  was 
tested  and  again  the  SPARCCS  application  did  not  function. 
As  such,  a  second  type  of  device  was  acquired,  a  Lenovo 
tablet  PC  with  the  Android  OS  version  2.2  was  tested  and 
SPARCCS  was  loaded.  On  this  device,  again,  the  application 
did  not  run.  Thus  it  was  demonstrated  that  while  the  Google 
Android  Application  Programming  Interface  (API)  used  to 
develop  SPARCCS  was  designed  to  develop  applications  that 
run  on  Android  2.2  or  greater,  this  was  not  the  case  with 
the  SPARCCS  application.  Based  on  the  controlled  testing  it 
can  be  concluded  that  SPARCCS  must  be  loaded  on  a  device 
that  supports  the  Android  OS  version  2.3  through  the  latest 
version  4.0.3. 
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2. 


Wireless  Technology  Testing 


The  wireless  technology  testing  program  was  developed 
to  test  the  specific  wireless  devices  to  be  used  in  the 
SPARCCS  field  testing.  This  testing  was  designed  to  ensure 
that  the  equipment  worked  properly  so  that  any  anomalies  or 
issues  identified  could  be  attributable  to  the  SPARCCS 
application  or  the  limitations  of  the  wireless  devices. 
Table  8  describes  the  testing  of  the  wireless  devices. 


Table  8.  Wireless  Device  Test  Details 


Test  Dates 

June  8,  June  14,  June  21,  June  23-27 

2012 

Test  Personnel 

Donna  Dulo,  NPS  Hastily  Formed  Network 

Group  Personnel  (HFN)  and  California 

Department  of  Forestry  and  Fire 

Protection  Personnel 

Equipment 

Hughes  9202  BGAN,  Persistent  Systems 

Wave  Relay  Radios,  WiMax  bridges, 

Verizon  Jetpack  with  Antenna 

Test  Software 

SpeedTest 

Test  Locations 

Outdoor  Test  Lab  in  Seaside,  CA,  Naval 

Postgraduate  School  Campus  (NPS) , 

CalFire  Campus. 

Test  Objectives 

To  determine  the  viability  of  the 

wireless  devices  to  be  used  in  the 

field  tests. 

The  details  of  the  individual  tests  are  presented  in 
Section  A  of  Appendix  A.  The  table  presents  the  specific 
qualitative  data  of  each  of  the  device  tests.  Note  the 
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specific  requirements  for  line  of  sight  in  some  of  the 
devices.  Figure  11  represents  one  of  the  impediments  to  the 
BGAN  as  noted  in  the  test  results. 


Figure  11.  Complete  Impediment  to  BGAN  Signal 

The  device  testing  at  the  outdoor  testing  range 
(Figure  12)  demonstrated  the  viability  and  functionality  of 
the  devices.  It  also  pointed  out  the  criticality  of  a  full 
satellite  line  of  sight  for  the  BGAN  device,  which  includes 
a  clear  and  unobstructed  skyline.  The  use  of  an  external 
compass  aided  in  the  more  rapid  and  accurate  pointing  of 
the  device,  which  helped  increase  the  data  rate  into  the 
WIFI  cloud.  The  testing  clearly  indicated  that  simple 
issues  such  as  a  window  screen,  a  fence  wire,  a  tree 
branch,  and  a  partial  view  of  the  satellite  could  cause  the 
BGAN  signal  to  degrade  or  drop. 
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Figure  12.  Outdoor  Testing  Range 


The  testing  also  indicated  a  clear  need  for  precise 
line  of  sight  between  the  WiMax  devices.  Even  a  slight 
deviation  from  line  of  sight  rendered  the  signal  weak  or 
lost.  The  wave  relay  radios  did  not  require  line  of  sight 


and  worked  well 

at 

any 

angle 

with 

the 

omni-directional 

antenna  attached 

to 

the 

radios . 

The 

need 

for  a  tethered 

power  supply  complicated  the  use  of  both  types  of  devices. 
The  mobility  of  the  devices  was  strictly  limited  to  the 
length  of  the  power  extension  cords  and  the  availability  of 
power  outlets.  The  use  of  a  car  lighter  power  inverter  was 
used  successfully;  however,  the  placement  of  the  vehicle  in 
the  wireless  mesh  limited  the  pointing  of  both  devices  and 
also  caused  clutter  in  the  network  range.  This  power 
situation  will  affect  the  use  of  these  devices  in  SPARCCS 
applications  that  require  mobility. 
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Figure  13.  Testing  the  BGAN  at  the  Outdoor  Testing  Range 

Finally,  the  outdoor  tests  demonstrated  that  the 
Jetpack  and  BGAN  devices  worked  well  in  the  outdoor 
environment  with  an  Android  device.  Note  the  use  of  the 

external  compass  in  Figure  13.  Through  the  use  of  the 

SpeedTest  software,  data  flows  were  demonstrated  to  show 
the  proper  functionality  of  the  devices  at  a  range  of  200 
feet . 

All  four  devices  were  shown  to  be  viable  and 

functional  and  ready  for  the  field  tests  with  the  SPARCCS 

applications.  The  limitations  of  the  devices,  as  discovered 
during  these  tests,  facilitated  more  accurate  field  test 
setups . 

3 .  Initial  Application  Testing 

The  testing  of  the  Android  devices  and  the  wireless 
devices  provided  a  solid  basis  for  the  testing  of  the 
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application  in  the  indoor  test  laboratory.  Through  properly 
functioning  devices,  the  application  was  tested  in  an 
isolated  manner  so  that  issues  could  be  spotted  and 
attributed  to  the  application  and  not  the  hardware.  The 
application  was  tested  exclusively  on  the  Jetpack  to  keep 
consistency  with  all  of  the  tests.  The  signal  strength  was 
at  a  consistent  "four  bars"  and  the  same  physical  location 
was  used  for  the  tests. 


Table  9.  Application  Tests 


Test  Dates 

May  15  -  June  2,  2012 

Test  Personnel 

Donna  Dulo 

Equipment 

All  listed  handheld  devices  of  Table 

5,  Verizon  Jetpack  with  Antenna, 

Toshiba  laptop,  Apple  iPad  1. 

Test  Software 

SPARCCS  Application 

Test  Locations 

Indoor  Test  Lab  in  Seaside,  CA 

Test  Objectives 

To  determine  the  initial  viability  of 

SPARCCS  application  to  be  used  in  the 

field  tests. 

The  application  testing  (Table  9)  tested  the  basics  of 
SPARCCS.  More  complete  functionality  was  tested  in  the 
field  tests.  As  a  result,  the  scope  of  the  initial 
application  tests  was  limited  to  basicfunctionality .  The 
handheld  device  and  the  Internet  application  were  both 
tested . 

A  specific  testing  paradigm  must  be  noted.  The  SPARCCS 
system  was  tested  as  a  complete  system.  Thus,  when  a 
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function  of  the  mobile  application  was  tested,  the  function 
was  followed  through  from  start  to  finish,  and  all  impacts 
of  the  function  were  evaluated.  If  a  function  produced 
changes  or  results  on  the  SPARCCS  Internet  application, 
then  the  testing  followed  the  function  from  its  start  on 
the  mobile  application  through  the  final  results  on  the 
Internet  application.  This  allowed  SPARCCS  to  be  tested  in 
as  complete  a  manner  as  possible,  to  assist  in  uncovering 
any  issues  with  functions  and  processes  as  they  impacted 
the  entire  SPARCCS  package. 

The  testing  of  the  application  went  well  over  the 
course  of  the  several  months.  Various  issues  were  noted  in 
the  tests  and  are  indicated  below.  Complete  qualitative 
field  notes  can  be  found  in  Section  A  of  Appendix  A. 

Testing  began  with  account  creation.  This  process 
worked  well  on  both  the  Internet  application  and  the  mobile 
application.  The  acronym  DORCCS,  which  was  the  name  of  an 

earlier  version  of  this  application  kept  popping  up  in 

message  screens,  as  shown  in  Figure  14.  The  code  should  be 

reviewed  to  eliminate  this  as  it  may  confuse  potential 
users  of  the  system.  A  second  issue  with  account  creation 
was  a  human  factors  issue  with  the  smart  phones.  It  was 
extremely  difficult  to  create  accounts  on  these  devices  as 
the  screens  are  small,  and  the  screen-based  keypads  make 
the  screen  even  smaller.  This  led  to  continual  errors  in 

data  input  and  the  need  to  restart  the  account  creation 
process.  This  issue  could  be  rectified  with  input  screens 
more  tailored  to  the  human  hand.  Otherwise,  on  the  tablets 
and  on  the  Internet  application,  account  creation  worked 
well  and  the  accounts  appeared  within  minutes  on  the 
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Internet  application  when  created  on  mobile  devices 
instantly  when  created  on  the  Internet  application. 


and 


Figure  14.  Example  of  Acronym  Mismatch 

The  mobile  and  Internet  applications  went  through 
extensive  login  testing,  which  was  a  basic  test  to  see  if 
accounts  created  were  viable  and  facilitated  login  into  the 
system.  All  devices  and  laptops  were  able  to  login  to  the 
system  on  both  applications  without  incident.  An  issue  did 
arise  during  testing  that  must  be  mentioned.  At  a  low 
signal,  the  SPARCCS  mobile  login  screen  was  not  painted 
properly  on  the  screen.  This  issue  arose  several  times 
during  testing.  This  was  tested  thoroughly  and  it  appears 
that  at  0-1  "bars,"  this  issue  occurs.  Figure  15,  reader's 
left  image.  With  1-5  "bars"  this  does  not  arise.  Figure  16 
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reader's  right  image.  Through  rigorous  testing  of  this 
issue  it  was  confirmed  that  this  is  a  signal  strength  issue 
only,  not  a  SPARCCS  application  issue.  This  should  be  noted 
in  SPARCCS  troubleshooting  documentation. 


Figure  15.  Demonstration  of  Proper  and  Incomplete  Login  Screen 

Presentation 


The  mission  creation  and  join  function  was  tested  on 
both  applications.  The  mission  creation  function  worked 
successfully  on  both  applications  in  most  instances. 
However,  in  some  cases  it  was  difficult  to  navigate  from 
the  user  screen  to  the  mission  creation  screen  and  back 
again.  For  example,  when  a  mission  was  created  or  joined. 
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the  back  navigation  brought  the  user  back  to  the  mission 
creation/ j  oin  screen  and  the  only  option  was  to  go  back 
into  the  mission  creation  page.  To  solve  this  issue  the 
application  had  to  be  restarted.  This  occurred  usually  when 
a  new  mission  was  created  and  then  a  user  tried  to  join  the 
mission  shortly  after.  This  appears  to  be  a  synchronization 
issue  between  the  device  creating  the  new  mission  and  the 
shared  database.  There  were  no  issues  on  the  Internet 
application . 

A  critical  part  of  the  SPARCCS  application  is  the 
Point  of  Interest  (POI)  function.  The  POI  function  was 
tested  on  all  devices  with  many  instances  on  each  device. 
This  function  worked  well  on  all  devices  without  incident. 
The  POIs  created  appeared  on  the  Internet  application  as 
input  and  the  Google  map  was  updated  with  the  POI  symbol  at 
the  relative  point  of  the  POI.  More  in  depth  testing  of  the 
POI  function  was  conducted  in  the  field  tests  and  will 
appear  later  in  this  chapter. 

A  central  function  of  the  SPARCCS  application  is  the 
photographic  capability,  as  it  captures  critical  images  of 
the  mission  to  send  back  to  command  and  control  for 
enhanced  situational  awareness.  The  photographic  function 
was  tested  on  all  devices.  In  each  instance  a  photograph 
was  taken  and  input  to  the  system.  In  all  cases  the 
photograph  was  successfully  transferred  and  visible  in  the 
Internet  application.  The  photographs  were  clear  and  of 
high  quality.  The  camera  icon  also  appeared  on  the  Google 
map  at  the  point  of  the  image  acquisition.  No  issues  were 
discovered  after  extensive  testing  of  this  function.  More 
in  depth  testing  of  the  photographic  function  was  conducted 
in  the  field  tests  and  will  appear  later  in  this  chapter. 
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The  GPS  integration  capability  of  SPARCCS  is  a  central 
feature  of  the  application  and  its  proper  functionality  is 
central  to  the  purpose  of  the  SPARCCS  application.  All 
devices  were  tested  with  their  GPS  capabilities  in  relation 
to  the  SPARCCS  application.  All  instances  with  the  Google 
maps  presented  precise  maps  with  the  current  user  icon  of 
SPARCCS  located  within  5  feet  of  the  actual  location  of  the 
device.  In  all  cases,  the  Google  map  was  refreshed  properly 
when  the  user  moved  location  with  the  device.  No  issues 
were  uncovered  concerning  Google  maps  or  GPS  issues.  During 
multiple  logins  of  users,  the  current  user  was  seen  as  an 
Android  icon  while  all  of  the  others  were  seen  as  green 
stars.  Figure  16. 


Figure  16.  User  Icon  and  Team  Member  Icons 

While  the  icons  demonstrated  accurate  positioning,  it 
was  impossible  to  tell  which  user  was  associated  with  a 

given  star  icon.  Thus,  the  screen  had  several  star  icons  on 
the  map,  but  individual  users  could  not  be  distinguished, 

which  may  be  a  critical  issue  in  the  real  world  where  team 

leaders  and  members  as  well  as  command  and  control 
personnel  need  to  know  exactly  "who  is  where"  during  the 
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mission.  This  issue  should  be  improved  in  future  versions 
of  the  software  with  different  icons  representing  different 
users . 

Finally,  the  SPARCCS  Internet  application  did  not  work 
on  the  U.S.  Army  network.  The  "us.army.mil"  domain 
specifically  blocks  all  Google  Appspot  applications.  This 
restriction  suggests  consideration  of  porting  the 
application  away  from  the  Google  cloud  should  it  ever  be 
considered  for  application  to  U.S.  Army  networks. 

Thus,  after  extensive  basic  testing  of  the  SPARCCS 
application,  the  application  was  ready  for  more  in  depth 
field-testing.  It  must  be  noted  that  the  above  tests  were 
for  basic  functionality.  More  in  depth  testing  was  reserved 
for  the  field  tests  that  follow  in  the  next  section.  In  the 
spirit  of  the  conservation  of  space,  advanced  features 
issues  that  were  noted  in  the  basic  application  testing  are 
brought  up  in  the  field-testing  as  to  avoid  repetition  of 
information.  The  following  section  describes  the  field 
tests  in  depth. 

B.  FIELD  TESTING 

The  field-testing  program  was  conducted  to  demonstrate 
and  test  the  SPARCCS  application  in  real  world  settings. 
The  field  tests  occurred  over  the  course  of  three  months. 
All  attempts  were  made  to  keep  the  scenarios  real  and 
applicable  to  potential  police,  fire,  and  military 
settings.  Each  field  test  scenario  was  treated  as  a 
discrete  entity,  no  previous  data  or  test  information  was 
used  to  ensure  that  sterile  tests  were  facilitated. 
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1. 


Camp  Roberts  Field  Test  I 


The  initial  Camp  Roberts  field  test  program  tested  the 
SPARCCS  application  in  the  context  of  a  real  world  Army 
scenario.  Its  main  goals  were  to  test  the  application  in  an 
Army  field  setting  in  the  test  range  of  Camp  Roberts, 
California,  a  California  National  Guard  installation  that 
specializes  in  the  training  of  National  Guardsmen.  Table  10 
describes  the  tests. 


Table  10.  Test  Details  of  the  Camp  Roberts  Initial  Testing 


Test  Dates 

July  12,  2012 

Test  Personnel 

Donna  Dulo,  John  Gibson  (driver) 

Equipment 

All  listed  handheld  devices  of  Table 

5,  Verizon  Jetpack  with  Antenna  and 

car  power  cord.  Pickup  truck. 

Test  Software 

SPARCCS  Application 

Test  Locations 

Camp  Roberts,  CA 

Test  Objectives 

To  determine  the  tracking  capability 

of  the  application,  to  determine  real 

world  functionality  (login,  mission 

creation,  mission  join,  POI  creation) 

of  the  application.  To  test  GPS 

accuracy  and  Google  map 

representation,  to  test  photographic 

capabilities  of  the  application  under 

commercial  wireless  connectivity. 

The  Camp  Roberts  testing  occurred  under  realistic 
circumstances,  in  summer  conditions.  The  day  was  clear  with 
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no  wind  and  the  temperature  was  between  88  and  92  degrees 
Fahrenheit.  Initial  testing  occurred  in  the  garrison 
staging  building  where  the  application  was  tested  with 
multiple  logons  of  users. 


Figure  17.  Test  Site  with  Multi-User  Logons 

As  can  be  seen  in  Figure  17,  three  users  are  logged 
onto  the  system  as  indicated  by  the  Android  icon  and  star 
icons.  These  are  the  three  users  that  were  a  part  of  the 
mission,  with  two  users  going  into  the  field  in  a  vehicle 
and  one  user  staying  behind  in  the  building. 
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Figure  18.  Higher  Level  Image  of  Field  Location 

Figure  18  presents  an  image  of  the  scenario  as  zoomed 
higher  above  the  scene.  The  mission  was  created  in  SPARCCS 
in  the  garrison  building.  The  process  went  smoothly; 
however  it  was  noted  that  the  mission  data  input  screen, 
shown  in  Figure  19,  is  highly  limited  in  terms  of  the 
ability  to  accommodate  text.  For  example,  the  Mission 
Miscellaneous  text  box  only  allowed  for  32  characters, 
which  is  not  enough  text  to  clearly  provide  input  for  the 
mission.  The  Mission  Description  text  box  had  the  same 
issue.  For  a  situational  awareness  application  data  input 
is  critical,  and  this  issue  should  be  rectified  to  allow 
for  more  extensive  text  input. 
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Figure  19.  Data  Capture  Screen  for  Mission  Creation 


Once  the  devices  and  the  application  were  tested  in 
the  garrison  building,  the  tracking  test  commenced.  The 
test  vehicle,  a  pickup  truck,  followed  a  convoy  of  Army 
vehicles  that  were  on  a  mission  to  test  a  Raven  Unmanned 
Aerial  Vehicle  (UAV)  in  the  field.  The  route  went  from  an 
elevation  of  approximately  100  feet  above  sea  level  to  a 
location  at  7  67  feet  above  sea  level.  One  device  was  left 
in  the  building  and  two  devices  were  located  in  the  pickup 
truck  with  the  Jetpack  system.  Figure  20,  left  image.  The 
Jetpack  antenna  was  mounted  on  the  top  of  the  truck  cab. 
Figure  21,  right  image. 
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Figure  20.  Vehicle  Setup  for  Tracking  Tests 

The  tracking  of  SPARCCS  was  precise  throughout  the 
testing.  The  user  icon  consistently  appeared  within  5  feet 
of  the  actual  location  of  the  vehicle  and  the  movement  of 
the  icon  was  in  real  time  and  in  relation  to  the  movement 
of  the  vehicle.  In  most  cases  the  icon  was  on  the  exact 
location  of  the  vehicle,  and  Figure  21  demonstrates  this, 
including  demonstrating  the  correct  side  of  the  road  of  the 
vehicle.  At  times  when  the  convoy  was  stopped,  the  icon 
location  was  measured  in  relation  to  the  actual  location  of 
the  vehicle  and  in  all  cases  the  icon  was  within  5  feet  of 
the  actual  location  and  in  most  cases,  the  distance  was 
almost  exact. 
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Figure  21.  Precision  Tracking  of  SPARCCS  at  Camp  Roberts 

During  the  test,  two  points  of  interest  were 
established  along  the  test  route.  SPARCCS  clearly  indicated 
these  points  at  the  location  of  their  creation,  within  10 
feet.  Issues  with  text  quantity  also  arose  with  the  POI 
screen,  as  discussed  previously  with  the  mission  creation 
text  boxes.  It  is  recommended  that  these  text  boxes  be 
increased  in  capacity  to  accommodate  more  extensive  field 
descriptions  of  the  POIs.  Figure  22  demonstrates  the 
locations  of  the  two  points  of  interest  that  were  created 
on  the  mission.  Note  that  the  user  located  at  the  garrison 
building  is  still  indicated.  Also  note  that  the  second 
device  (user)  that  is  logged  into  the  system  that  is  in  the 
tracking  vehicle  with  the  first  device  is  not  indicated. 
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This  was  another  issue  that  was  discovered  in  this 


field  test.  During  the  tracking  exercise,  only  one  user  is 
tracked  as  moving  with  the  vehicle.  The  other  user's  star 
is  not  visible.  Only  the  stationary  user's  star  is  visible 
during  the  tracking  test.  This  is  a  major  issue  to  be 
rectified;  as  all  users  should  be  seen  on  the  map  as  each 
user  moves  throughout  the  mission.  The  incomplete  picture 
of  users  has  the  potential  to  be  detrimental  to  mission 
success,  especially  in  life  safety  operations. 


Figure  22.  POIs  and  User  Locations  at  Camp  Roberts 

The  SPARCCS  Internet  application  worked  well  during 
the  testing.  All  data  captured  by  the  mobile  devices  was 
successfully  transferred  to  the  Internet  application. 
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Figure  23  demonstrates  the  Internet  application  and 
corresponding  mission  location,  team  leader,  mission 
creator,  and  data  gathered  on  the  mobile  application. 
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Figure  23. 


Camp  Roberts  Mission  Main  Screen  on  Internet 

Application 


Finally,  the  photographic  and  archival  functions  of 
the  SPARCCS  application  were  tested.  At  the  second  POI  a 
series  of  photographs  were  taken  of  a  Raven  UAV  launch.  All 
photographs  were  successfully  transferred  to  the  Internet 
application  with  the  correct  data  associated  with  them. 
Figure  24  depicts  one  of  the  photographs  in  the  SPARCCS 
application.  As  can  be  seen,  the  photograph  is  clear  and  of 
a  high  quality.  The  data  associated  with  the  photograph  is 
accurate.  The  photograph  is  associated  with  the  correct  POI 
and  is  precisely  time-stamped  in  association  with  the 
mission . 
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Figure  24.  SPARCCS  Photographic  Imaging  Function  with  POI 


Overall,  the  Camp  Roberts  initial  field  test  was  a 
success.  It  demonstrated  the  power  and  precision  of  SPARCCS 
as  well  as  some  issues  that  need  to  be  rectified  in  future 
versions  of  the  software.  The  testing  demonstrated  that 
SPARCCS  is  a  viable  application  for  Army  field  operations. 
The  issues  discovered  during  this  test  will  not  be 
mentioned  further  in  the  following  field  tests  for  sake  of 
brevity . 

2 .  Camp  Roberts  Field  Test  II 

A  second  round  of  testing  was  conducted  at  Camp 
Roberts,  CA,  in  August  of  2012.  The  purpose  of  this  round 
of  tests  was  to  test  the  Wave  Relay  and  WiMax  devices  in  a 
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real  world  scenario  involving  a  simulated  disaster 
operation  over  a  larger  operational  range.  Table  11 
discusses  the  details  of  the  testing. 


Table  11.  Camp  Roberts  II  Testing  Details 


Test  Dates 

August  16,  2012 

Test  Personnel 

Donna  Dulo, (setup  &  test)  Marcelo 

Perfetti  (setup) ,  Mark  Simmons  (setup) 

Equipment 

Galaxy  Tablet,  Toshiba  Tablet,  Droid 

X,  Wave  Relay  Devices,  WiMax  Devices, 

Omni-directional  antennae,  directional 

antennae,  heavy  dytu  tripods,  Toshiba 

laptop,  author' s  privately  owned 

vehicle 

Test  Software 

SPARCCS  Application 

Test  Locations 

Camp  Roberts,  CA 

Test  Objectives 

To  compare  and  contrast  the 

performance  of  Wave  Relays  and  WiMax 

devices  as  communications  bridges  and 

to  demonstrate  SPARCCS  on  those 

networks,  to  demonstrate  the  Wave 

Relay  device  as  a  wireless  hotspot  on 

a  long  distance  Wave  Relay  network. 

The  testing  was  a  part  of  the  larger  event  on  Camp 
Roberts,  the  Joint  Interagency  Field  Exploration  Research 
and  Experimentation  for  Local  and  International  Emergency 
First  Responders  (JIFX  RELIEF) .  The  purpose  of  the  overall 
test  was  to  test  the  viability  of  the  Wave  Relay  and  WiMax 
devices  over  long-range  communications.  Our  testing  used 
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these  devices  for  SPARCCS  as  an  additional  set  of  testing. 
The  equipment  was  set  up  over  the  entire  Camp  Roberts 
installation,  spanning  over  8  miles.  The  SPARCCS  testing 
was  conducted  separately  and  independently  by  the  author 
after  the  devices  were  set  up  by  the  Naval  Postgraduate 
School  team.  The  outdoor  temperature  was  between  85  and  95 
degrees  Fahrenheit  throughout  the  day. 

The  scenario  consisted  of  a  disaster  site,  two 
communications  relay  hills,  and  a  command  headquarters 
site.  Figure  25  demonstrates  the  sites  with  photographs  and 
the  paths  of  communications  between  the  sites. 


Relay  Point  2 


Figure  25.  The  Communication  Sites  at  Camp  Roberts 

Each  site  had  a  Wave  Relay  setup  and  a  WiMax  setup, 
independent  of  each  other,  stationed  about  5  feet  apart 
from  each  other  on  heavy-duty  tripods.  Each  network  had 
connectivity  through  a  Hughes  INMARSAT  Ku-band  BGAN  device 
that  was  located  and  setup  at  the  disaster  site.  Figure  26 


demonstrates  the  setup  of  the  communication  devices. 
Appendix  A  Section  B  presents  selected  screenshots  of  setup 
screens  of  the  devices . 


Figure  26.  Camp  Roberts  Disaster  Scenario  Setup 

The  devices  were  configured  at  each  site  and  then 
adjusted  over  a  period  of  two  hours  to  align  them  properly. 
The  WiMax  devices  required  precision  line  of  sight  while 
the  Wave  Relay  devices  required  to  be  pointed  in  the 
general  direction  of  the  next  device.  Figure  27 
demonstrates  the  setup  of  the  devices.  This  particular 
setup  was  at  the  headquarters  site  located  on  McMillan 
Airfield  at  Camp  Roberts. 
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Figure  27.  WiMAX  Device  (right)  and  Wave  Relay  (left) 


During  the  initial  setup  testing  it  was  determined 
that  the  omni-directional  antenna  was  more  ideal  and 
presented  significantly  greater  performance  than  the 
directional  antenna  on  the  Wave  Relays.  Figure  28  depicts 
the  omni-directional  antenna  on  the  Wave  Relay  device. 
Thus,  all  Wave  Relays  were  equipped  with  omni-directional 
antenna  for  the  duration  of  the  testing. 


Figure  28.  Wave  Relay  with  Omni-Directional  Antenna 
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The  device  ranges  covered  extensive  distances  between 
the  various  sites.  Figure  29  shows  the  various  distances 
covered  by  the  devices . 


Figure  29.  Disaster  Scenario  Ranges 


The  readings  on  the  devices  were  taken  by  driving  up 
to  the  site  in  a  privately  owned  vehicle  approved  by  the 
Camp  Roberts  range  authority,  and  plugging  in  a  laptop  to 
take  the  readings.  Appendix  A  has  images  that  demonstrate 
the  readings  screens.  There  were  2  hours  between  the  2  sets 
of  readings.  It  took  approximately  50  minutes  to  do  a  set 
of  readings  by  driving  to  each  site.  Table  12  contains  the 
data  gathered  from  the  two  sets  of  readings  taken  at  the 
sites . 
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Table  12.  Camp  Roberts  II  Bridge  Data  Speeds  in  Mbps 


Location 

Wave  Relay 

Round  1 /Round  2 

WiMax 

Round  1 /Round  2 

Disaster  Site 

5.89  /  5.32 

5.76  /  5.01 

Relay  Hill  1 

3.45  /  3.88 

3.21  /  3.13 

Relay  Hill  2 

3.73  /  3.64 

2.43  /  1.98 

C2  HQ 

2.68  /  2.51 

1.56  /  1.34 

As  indicated  by  the  data,  the  Wave  Relay  had  stronger 
performance  as  a  network.  The  signal  degraded  less  and  the 
overall  delivery  to  the  headquarters  site  was  1.12  Mbps  and 
1.17  Mbps  greater  on  the  two  test  rounds.  This  is  a  visible 
indicator  of  the  Wave  Relay  system' s  higher  sustainment  of 
the  signal  over  the  course  of  9.1  miles  of  communications 
hops,  the  6.6-mile  link  being  the  constraining  factor. 

Basic  SPARCCS  functionality  was  tested  at  the 
headquarters  site  by  using  a  CAT  5  cable  and  plugging  it 
into  a  Toshiba  laptop  and  setting  up  a  small  wireless 
cloud.  Through  basic  functional  tests,  SPARCCS  performed 
equally  on  both  the  Wave  Relay  and  the  WiMax  devices.  Basic 
tests  such  as  POI  creation,  GPS  functionality,  data  input, 
and  photographic  tests  were  all  successful  on  both  networks 
with  no  difference  in  performance  that  was  noticeable.  Thus 
while  the  WiMax  device  network  yielded  a  smaller  data  rate, 
the  difference  was  not  large  enough  to  show  an  appreciable 
performance  increase  in  SPARCCS  on  the  Wave  Relay  network. 


The  final  test 

was  to 

demonstrate 

the  Wave 

Relay 

device 

as  a  wireless 

hotspot . 

This  was  not 

possible 

on  the 

WiMax 

device  as  it 

is  a  bridging  device 

only.  A 

second 

radio 

was  configured 

on  the 

device  and 

a  second 

omni- 

directional  antenna  was  installed.  The  SPARCCS  application 

was  then  connected  to  the  network  successfully  and  a  mock 

mission  around  McMillan  airfield  was  conducted. 
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Figure  30.  McMillan  Airfield  Photographic  Points 

Figure  30  demonstrates  the  two  photographic  points  at 
each  end  of  the  airfield.  Figure  31  demonstrates  the  POI 
created  at  the  east  end  of  the  runway.  The  screenshots  also 
demonstrate  the  exceptional  range  of  the  WiFi  radio  on  the 
Wave  Relay,  which  spanned  the  entire  length  of  the  garrison 
at  the  airfield.  The  test  also  demonstrated  that  a  WIFI 
cloud  could  be  created  at  the  final  headquarters  node  the 
of  9.1  miles  of  hops  in  a  Wave  Relay  network  operating  off 
of  a  BGAN  Internet  signal. 
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Figure  31.  POI,  Photograph,  and  User  Location  Demonstration 

Overall,  the  Camp  Roberts  field  tests  demonstrated  the 
power  of  both  the  Wave  Relay  radios  and  the  WiMax  devices 
as  long  range  bridging  devices  and  the  power  of  the  Wave 
Relay  to  form  a  long  range  network  and  to  form  WIFI 
hotspots  that  allow  applications  like  SPARCCS  to  function 
fully.  The  combination  of  SPARCCS  and  these  devices 
provides  a  viable  communications  option  for  emergency 
responders  and  military  personnel  who  require  situational 
awareness  over  a  long  distance. 

3.  MIRA  Marina  Campus  Tests 

The  Monterey  Institute  for  Research  in  Astronomy 
permitted  SPARCCS  testing  at  their  Marina,  CA  site.  Their 
site  is  a  four  building  campus  with  a  large  main  building, 
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two  shop  buildings  and  an  observatory  building.  The  campus 
is  260  feet  by  295  feet  and  is  located  on  the  former  Ft. 
Ord.  This  scenario  was  used  to  demonstrate  the  SPARCCS 
application  in  an  urban  like  setting  where  responders  must 
transverse  between  multiple  buildings.  The  tests  were 
conducted  in  six  different  sessions  over  the  course  of  a 
three-week  period.  Table  13  provides  the  details  of  the 
tests.  The  MIRA  campus  is  depicted  in  Figure  32. 


Table  13.  MIRA  Marina  Campus  Test  Details 


Test  Dates 

Aug  1  -  24,  2012 

Test  Personnel 

Donna  Dulo,  Tami  Huntley,  Ada  Hynes 

Equipment 

All  listed  handheld  devices  of  Table 

5,  Verizon  Jetpack  with  Antenna,  BGAN 

antenna 

Test  Software 

SpeedTest,  SPARCCS  Application 

Test  Locations 

MIRA  Marina  Campus,  CA 

Test  Objectives 

To  determine  the  ranges  of 

communications  of  the  Jetpack  device 

and  the  BGAN  device  and  the 

performance  of  SPARCCS  on  each  in  a 

campus  setting. 
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Figure  32 .  The  Marina  MIRA  Campus 


The  MIRA  campus  was  drawn  on  a  map  and  17  different 
test  locations  were  established  as  depicted  in  Figure  33. 
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These  were  the  points  where  data  readings  were  taken.  Over 
the  course  of  six  different  sessions  in  a  three-week  time 
period,  the  BGAN  and  Jetpack  were  set  up  and  tested  through 
readings  at  each  of  these  points.  Appendix  A  Section  C 
contains  the  data  for  a  test  run  for  each  of  the  devices. 
Four  test  runs  were  conducted  on  each  device  and  the 
results  were  similar.  Due  to  space  constraints,  only  one 
set  of  each  was  selected  for  inclusion  in  the  Appendix. 
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Figure  33.  MIRA  Campus  Data  Sample  Points 


The  Verizon  Jetpack  operated  consistently  between  3-4 
"bars."  The  weather  was  consistent  among  all  test  dates. 
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overcast  and  between  55  and  65  degrees  F.  The  devices  were 
setup  in  the  each  location,  as  illustrated  in  Figure  34. 


Figure  34.  Testing  Point  1  with  the  BGAN 

The  test  data,  as  presented  in  Appendix  A,  clearly 
indicated  that  the  Verizon  Jetpack  covered  the  campus 
easily  with  strong  signals  throughout  the  campus.  Of  note 
was  the  high  reading  levels  around  the  metal  building  and 
the  exceptionally  high  readings  next  to  the  observatory 
antenna.  The  wireless  signal  covered  the  backside  of  the 
buildings,  although  in  a  degraded  state  at  the  far  corner 
of  the  concrete  building. 
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With  the  user  device  directly  connected  to  the  BGAN, 
the  signal  to  the  device  was  weakened  or  terminated  the 
test  device  was  not  in  the  line  of  sight  of  the  BGAN 
antenna.  Signals  were  received  approximately  4-5  feet 
around  the  corner  of  the  buildings  but  beyond  that  no 
signals  were  received.  This  is  a  strong  indicator  that  the 
BGAN  may  not  be  ideal  for  other  than  line  of  sight.  The 
BGAN  signal  did  stretch  the  full  length  of  the  campus,  but 
only  in  line  of  sight.  Figure  35  demonstrates  the  BGAN 
signal  blackout  zones  on  the  campus. 


Concrete  I  uilding 
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The  SPARCCS  application  was  tested  at  all  points  of 


the  campus  including  blackout  points.  Figure  36  depicts  the 
mission  on  the  headquarters  application.  The  results  of  the 
tests  suggest  that  as  long  as  there  was  a  channel  of  at 
least  200  Kbps,  the  SPARCCS  application  worked  fine, 
including  POI  creation  and  data  entry,  photography,  and  GPS 
mapping.  The  application  did  not  work  with  data  rates  less 
than  100  Kbps  and  intermittent  issues  like  partial  screen 
painting,  application  crashes,  and  application  freezing 
occurred  in  the  range  of  approximately  100  to  200  Kbps, 
consistently.  When  the  signal  was  above  200  Kbps  on  either 
the  BGAN  or  Jetpack  network,  there  was  no  noticeable 
difference  in  performance  between  the  two  wireless  reach- 
back  methodologies. 


Figure  36 


MIRA  Mission  on  the  Headquarters  Application 
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The  only  unique  application  issue  that  was  noted  was 
the  POI  location  notes  textbox,  like  the  mission  textboxes 
described  earlier,  which  could  only  accommodate  a  limited 
amount  of  text,  as  demonstrated  in  Figure  37. 


Initial  depot  of  supply  to  be  sent  to  MIRA  for  special  mission  this  evening. 
The  supplies  are  classified  and  will  be  stored  under  guard  Only 
authorized  personnel  will  have  access  to  the  inventory  list  and  the  list 
will  be  kept  in  the  headquarters  safe  at 
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Figure  37.  SPARCCS  POI  Creation 


The  MIRA  campus  tests  demonstrated  the  power  of  the 
Jetpack  and  commercial  wireless  network  for  an  urban 
SPARCCS  application.  The  BGAN  was  shown  to  be  less  than 
optimal  for  situations  that  have  other  than  unobstructed 
line-of-sight .  Overall,  the  tests  demonstrated  the 
viability  of  SPARCCS  for  an  urban  situation  and  the 
performance  capabilities  of  cellular  and  BGAN  WIFI 
extensions . 

4 .  MIRA  Chews  Ridge  Mountain top  Tests 

The  Monterey  Institute  for  Research  in  Astronomy  Chews 
Ridge  observatory  campus,  located  at  5000  feet  elevation  in 
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the  mountains  near  Big  Sur  was  chosen  as  a  site  to  test  the 
remote  capabilities  of  the  BGAN  antenna  and  the  ability  of 
SPARCCS  to  operate  on  the  WIFI  cloud  generated  by  the  BGAN. 
The  site  is  remote,  as  shown  in  Figure  38,  and  there  are  no 
commercial  cellular  signals  available.  Table  14  provides 
the  details  of  the  tests. 


Table  14.  MIRA  Chews  Ridge  Campus  Tests  Details 


Test  Dates 

July  28,  2012 

Test  Personnel 

Donna  Dulo,  Tami  Huntley,  Elizabeth 

Cameron 

Equipment 

Galaxy  and  Toshiba  tablet  PCs,  Droid  X 

Test  Software 

SpeedTest,  SPARCCS  Application 

Test  Locations 

MIRA  Chews  Ridge  Campus,  CA 

Test  Objectives 

To  determine  the  capability  of  the 

BGAN  antenna  and  the  SPARCCS 

application  to  operate  in  a  remote 

site  at  an  elevation  of  5000  feet. 
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Figure  38.  View  from  MIRA  Site  Demonstrating  Remote  Location 

The  BGAN  was  setup  at  the  far  end  of  the  MIRA  campus . 
A  test  range,  depicted  in  Figure  39,  with  cones  laid  out 
every  50  feet  was  established.  Data  readings  were  taken  at 
all  points  in  the  test  range.  Four  rounds  of  tests  were 
taken.  All  four  rounds  produced  similar  results.  Appendix  A 
Section  D  provides  the  data  for  one  of  the  test  rounds.  The 
elevation  ranged  from  5003  to  5015  feet  with  an  increase  in 
elevation  near  the  300  feet  marker.  This  elevation  can  be 
seen  in  Figures  39  and  40. 
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Figure  39.  BGAN  Test  Range 

The  data  indicates  that  the  BGAN-supported  WiFi 
produced  a  strong  signal  for  up  to  300  feet,  where  soon 
after  the  signal  became  unstable.  This  is  the  exact  point 
of  the  increase  in  elevation.  After  the  300  feet  marker, 
the  WiFi  cloud  generated  by  the  BGAN  device  was  no  longer 
in  the  direct  line  of  sight  of  the  test  device.  After  350 
feet  data  transfer  was  no  longer  possible. 
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Figure  40.  BGAN  Test  Range  Far  End 

The  SPARCCS  application  was  tested  at  all  points  of 
the  test  range.  Tests  included  mission  creation,  POI 
creation,  POI  data  entry  and  GPS  capabilities.  Photos  were 
also  taken  through  the  SPARCCS  application.  The  application 
worked  well  at  distances  between  0  through  250  feet.  Issues 
began  to  develop  when  the  ground  began  its  upslope.  These 
included  application  halts,  application  freezing,  and 
failure  to  paint  SPARCCS  pages  on  the  device  screen.  The 
application  worked  well  on  all  three  test-devices.  Figure 
41  illustrates  the  GPS  test.  Note  the  remote  location  of 
the  site. 
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SPARCCSm 


Figure  41.  Remote  MIRA  Location  on  SPARCCS  GPS  Map 


The  SPARCCS  application  worked  as  if  it  were  in  an 
urban  area.  No  differences  in  performance  were  noted 
despite  the  remote  location  of  the  test.  This  demonstrates 
that  SPARCCS  performance  is  linked  to  the  strength  of  the 
wireless  signal  not  the  source  of  the  signal  or  application 
of  the  signal  to  a  specific  scenario.  The  tests 
demonstrated  that  SPARCCS  is  a  viable  tool  for  missions  in 
the  wilderness  for  both  the  military  and  wilderness 
firefighting.  The  only  issue  would  be  the  BGAN  line  of 
sight,  which  must  be  carefully  considered  when  choosing  the 
BGAN  for  a  scenario  like  the  MIRA  Chews  Ridge  campus. 


92 


5. 


Ft.  Ord  Concrete  Range  Field  Tests 


The  purpose  of  the  concrete  test  range  tests  was  to 
test  the  range  and  performance  of  the  Jetpack  and  the  BGAN 
on  a  flat,  unobstructed  course  to  determine  the  range  of 
operation  of  the  devices,  and  the  performance  of  SPARCCS  on 
the  outer  edges  of  device  capability.  Additionally,  the 
testing  of  the  power  capabilities  of  the  devices  was 
determined  to  give  an  indication  of  how  long  each  device 
could  support  a  SPARCCS  operation  without  being  re-charged. 


Table  15.  Concrete  Range  Test  Details 


Test  Dates 

July  10  -  14,  2012 

Test  Personnel 

Donna  Dulo,  Tami  Huntley,  Ada  Hynes 

Equipment 

All  listed  handheld  devices  of  Table 

5,  Verizon  Jetpack  with  Antenna,  BGAN 

antenna 

Test  Software 

SpeedTest,  SPARCCS  Application 

Test  Locations 

Former  Ft.  Ord,  CA 

Test  Objectives 

To  determine  the  linear  ranges  of 

communications  of  the  Jetpack  device 

and  the  BGAN  device  and  the 

performance  of  SPARCCS  on  each. 

Table  15  describes  the  details  of  the  testing  which 
occurred  over  a  five  day  period  on  a  flat  concrete  test 
range  on  the  former  Ft.  Ord.  Figure  42  shows  a  test  team 
member  measuring  out  the  test  range  with  the  ranging  wheel . 
Chalk  lines  were  used  in  conjunction  with  orange  safety 
cones  to  ensure  that  each  test  day  had  the  same  course  of 
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measurement.  The  range  went  from  0  to  1500  feet  with  a 
measuring  point  (cone)  every  100  feet. 


Figure  42.  Measuring  Out  the  Test  Range 


A  set  of  data  from  two  test  runs  can  be  found  in 
Appendix  A.  As  can  be  seen,  the  BGAN  device,  with  its  built 
in  WiFi  cloud,  had  a  longer  distance  of  viable  signals, 
1100  feet  as  compared  to  the  Jetpack,  which  had  a  viable 
range  of  700  feet.  The  BGAN  significantly  outperformed  its 
manufacturer's  range  of  a  100  feet  WIFI  cloud  (Hughes, 
2012) . 

The  Jetpack  had  a  strong  signal  in  all  of  the  test 


runs  right 

up  until  it  hit 

the 

edge 

of 

its 

advertised 

range .  As 

the  data  indicates. 

at 

700  feet 

its 

performance 

was  strong, 

,  however  at  800  feet  it  had 

an 

initial  reading 

of  no  signal  with  a  subsequent  set  of  readings 
significantly  lower  than  previous  distances.  No  signal  at 
all  was  received  at  900  feet  and  greater. 
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The  BGAN  with  its  built  in  WiFi  cloud  had  a  somewhat 
inconsistent  performance  in  the  range  tests.  Notice  a 
sudden  drop  at  400  feet  in  the  test  data  presented  in 
Appendix  A  then  a  large  rise  at  500  feet.  Also,  during 
gusts  of  wind,  which  were  prevalent  on  this  flat  range,  the 
BGAN  signal  was  significantly  higher.  The  BGAN  had  solid, 
consistent  performance  up  to  1100  feet  before  the  signal 
began  to  degrade . 

SPARCCS  was  tested  on  both  platforms  at  the  outer 
range  of  connectivity.  The  application  performed  fully  with 
channels  over  150  Kbps  and  performed  marginally  for 
channels  between  100  and  149  Kbps.  Performance  indicators 
of  a  poor  signal  were  slow  application  response  times,  slow 
or  improper  screen  painting,  and  inability  to  send  pictures 
or  create  a  mission  or  POI .  Additionally,  SPARCCS  tended  to 
halt  when  there  was  a  significant  change  in  signal;  for 
example,  with  the  BGAN,  between  300  and  500  feet,  there 
were  several  application  halts  when  the  upload  speeds  had  a 
wide  variance.  In  the  areas  with  consistent  signals  over 
150  Kbps  the  SPARCCS  application  worked  well  in  all  aspects 
of  functionality. 

A  final  round  of  testing  concerned  the  battery  power 
of  the  two  wireless  devices.  In  the  field  it  is  critical  to 
understand  the  power  limitations  of  the  devices  so  that 
mission  planning  can  accommodate  recharging  sessions  or 
device  replacement.  Table  16  presents  the  data  for  battery 
testing.  The  rounds  were  conducted  on  different  days  and 
the  devices  were  in  operating  mode  during  the  tests. 
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Table  16.  Device  Battery  Life 


Device 

Round 

Charge  Time  in 

Operation 

BGAN 

1 

5  hours  23  min 

BGAN 

2 

5  hours  35  min 

Jetpack 

1 

2  hours  4  min 

Jetpack 

2 

2  hours  12  min 

As  can  be  seen,  the  BGAN  has  a  significantly  longer 
life  than  the  Jetpack.  This  particular  BGAN  device  had  a 
battery  only  3  months  old,  so  it  may  be  considered  new. 
However,  the  manufacturer  (Hughes,  2012)  claims  the  battery 
life  to  be  6  hours,  so  this  device  fell  short  of  this 

claim . 

The  Jetpack  had  a  shorter  battery  life.  However,  its 
recharge  time  was  only  30  minutes  compared  to  several  hours 
for  the  BGAN.  In  addition,  the  Jetpack  could  be  charged 

easily  with  a  car's  lighter  adapter,  as  opposed  to  the  120 
volt  requirement  for  the  BGAN,  so  the  jetpack  could  be 
brought  back  online  faster  than  the  BGAN. 

Overall,  the  concrete  range  tests  demonstrated  the 

distance  of  signals  and  battery  limitations  of  the  jetpack 
and  BGAN;  critical  information  when  planning  a  SPARCCS 
mission.  The  data  collected  serves  as  a  guide  to  mission 

planning  and  selecting  the  proper  device  for  the  particular 
mission  tasking. 

6.  Ft.  Ord  Wildland  Scenario  Field  Tests 

The  Ft.  Ord  Wildland  field  tests  were  similar  to  the 
concrete  test  range  tests  with  the  difference  being  natural 
shrubbery  occurring  along  the  test  path.  In  these  tests,  a 
1500  feet  test  range  was  measured  out  through  a  patch  of 
shrubbery  and  trees  to  determine  signal  strength  and  decay 
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for  missions  involving  wildland  operations  for  military  or 
emergency  personnel.  Table  17  describes  the  details  of  the 
testing . 


Table  17.  Wildland  Range  Tests  Details 


Test  Dates 

July  15-21,  2012 

Test  Personnel 

Donna  Dulo,  Tami  Huntley,  Ada  Hynes 

Equipment 

All  listed  handheld  devices  of  Table 

5,  Verizon  Jetpack  with  Antenna,  BGAN 

antenna 

Test  Software 

SpeedTest,  SPARCCS  Application 

Test  Locations 

Former  Ft.  Ord,  CA 

Test  Objectives 

To  determine  the  ranges  of 

communications  of  the  Jetpack  device 

and  the  BGAN  device  and  the 

performance  of  SPARCCS  on  each  in  a 

wildland  setting  with  trees  and 

shrubbery  as  obstacles. 

The  tests  were  conducted  in  a  large,  open  woodland 
area  on  the  former  Ft.  Ord  during  a  7-day  period.  The  test 
setup  was  identical  to  the  concrete  range  with  points  of 
measurement  every  100  feet.  The  area  was  relatively  flat  so 
that  elevation/line  of  sight  would  not  be  a  factor  in  the 
tests.  Figure  43  shows  the  test  range. 
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The  data  from  the  wildland  tests  can  be  found  in 
Appendix  A.  The  data  was  significantly  different  than  the 
open  flat  range  test  data.  The  BGAN  saw  performance  decay 
once  medium  density  shrubbery  was  present.  These  were  low 
hanging  trees,  as  depicted  in  Figure  43.  As  can  be  seen, 
the  trees  did  not  have  large  trunks,  just  low  handing 
branches  that  filled  the  area  with  foliage.  Once  the 
foliage  density  increased  the  BGAN-provided  WiFi  signal 
rapidly  decayed.  Thus,  through  200  feet  of  shrubbery  the 
signal  went  from  "strong"  to  "nothing, "  indicating  that 
foliage  obstacles  can  seriously  impact  the  BGAN  unit's  use 
in  the  woods.  Thus,  it  can  be  concluded  that  the  BGAN' s 
WiFi  signal  will  degrade  upon  the  commencement  of  medium 
density  shrubbery  and  will  completely  degrade  soon  after. 

The  Jetpack,  conversely,  demonstrated  little 

difference  between  its  concrete  test  range  readings  and  its 
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wildland  readings.  Both  tests  demonstrate  a  700  feet  solid 
signal  radius.  This  indicates  that  the  shrubbery  and 
foliage  did  not  have  a  significant  performance  impact  on 
the  Jetpack  signal.  This  is  a  critical  piece  of  information 
as  it  indicates  that  the  Jetpack  is  a  more  viable  option 
for  communications  in  a  wooded  area.  It  also  raises  the 
issue  of  frequency  range,  as  higher  frequencies  are  more 
sensitive  to  foliage  than  lower  frequencies.  The  use  of 
SPARCCS  in  a  wildland  setting  should  be  carried  out  with  a 
device  with  the  lowest  frequency  range. 

SPARCCS  performed  fully  with  signals  over  150  Kbps  and 
performed  marginally  between  100  and  149  Kbps  as  also 
indicated  in  the  concrete  test  range  tests.  Performance 
indicators  for  a  poor  signal  were  slow  application  response 
times,  slow  or  improper  screen  painting,  inability  to  send 
pictures,  and  inability  to  create  a  mission  or  POI .  With 
the  Jetpack,  the  application  performed  well  in  the  midst  of 
the  foliage  within  the  700  feet  range,  as  opposed  to 
performance  with  the  BGAN,  which  showed  application  issues 
within  the  initial  200  feet  of  the  foliage  line. 

Thus,  the  wildland  tests  clearly  indicate  that  the 
selection  of  a  BGAN  or  Jetpack  must  rest  on  the  density  of 
the  area  of  the  mission.  It  will  also  rest  on  the  signal 
for  the  cellular  service.  During  these  tests  the  jetpack 
had  a  3-4  bar  signal  strength.  However,  in  remote  locations 
the  BGAN  may  be  the  only  option.  In  this  case,  care  should 
be  made  to  plan  out  the  BGAN  range  with  obstacles  in  mind. 

7 .  Vehicle  Distance  Tracking  Tests 

The  vehicle  tracking  tests  were  a  more  lengthy 
extension  of  the  initial  Camp  Roberts  tests.  The  purpose  of 
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these  tests  was  to  demonstrate  the  long-range  capabilities 
of  SPARCCS  to  support  a  mission  that  required  responders  to 
travel  a  long  distance.  Table  18  describes  the  details  of 
the  tests. 


Table  18.  Vehicle  Distance  Tracking  Details 


Test  Dates 

July  18,  2012 

Test  Personnel 

Donna  Dulo,  Tami  Huntley 

Equipment 

Galaxy  Tablet  and  Droid  X 

Test  Software 

SpeedTest,  SPARCCS  Application 

Test  Locations 

Monterey  County,  CA 

Test  Objectives 

To  determine  the  tracking  ability  of 

SPARCCS  on  a  long  range  mission. 

The  tests  were  conducted  on  two  California  highways, 
from  Ft.  Ord  to  Gilroy,  California.  Several  stops  were  made 
along  the  way  to  take  readings .  Stops  were  made  in  parking 
lots  to  be  able  to  clearly  measure  the  precision  of  the 
Google  mapping  in  SPARCCS  through  parking  lot  lines.  The 
tests  took  two  hours  to  complete. 

The  tests  consisted  of  three  devices  logged  onto  the 
same  mission  using  Jetpack  connectivity  with  an  external 
antenna  mounted  on  the  exterior  of  the  vehicle.  The  devices 
were  maintained  by  the  vehicle  passenger.  Screenshots  were 
taken  at  specific  test  points.  The  tracking  of  the  vehicle 
was  observed  by  the  tester  in  the  passenger  seat  to  ensure 
accuracy  and  precision  with  the  mapping.  Figure  44 
demonstrates  the  initial  location  of  the  tests.  Note  the 
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time  stamps  on  the  screenshots  that  indicate  time  travelled 
during  the  testing.  The  test  drive  began  at  approximately 
8:30  PM. 


Figure  44.  Initial  Location  with  Three  Devices  on  the  Mission 

At  approximately  8:55  the  vehicle  arrived  in 
Prunedale,  CA  and  a  set  of  screen  shots  were  taken,  as 
shown  in  Figure  45.  SPARCCS  tracked  perfectly  from  the 
beginning  of  the  trip  to  this  site.  Note  that  the  user  icon 
is  in  the  exact  parking  spot  as  the  test  vehicle.  The  POI 
flag  was  slightly  off  location  by  about  twenty  feet.  The 
POI  information  was  collected,  input,  and  displayed  well  on 
the  SPARCCS  Internet  application.  The  SPARCCS  application 
worked  fluidly  through  the  entire  leg  of  the  trip  with  no 
breaks  in  connectivity  or  application  freezes  or  halts. 
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Figure  45.  First  Test  Location 


At  approximately  9:30  PM  the  test  vehicle  arrived  in 
Gilroy,  CA,  with  all  three  devices  still  tracking  with  no 
breaks  in  connectivity  for  any  of  the  three.  Figure  46 
demonstrates  the  screenshots  from  this  location.  Note  that 
in  this  instance  the  user  icon  and  the  POI  icon  are  in  the 
exact  location  of  the  vehicle  in  the  parking  lot. 

The  tests  indicated  that  the  SPARCCS  application  is 
robust  in  its  tracking  ability  over  long  ranges.  The  tests 
also  indicated  that  the  mapping  of  the  icons  is  quite 
precise,  even  after  over  an  hour  of  operation  on  a  mission. 
The  only  issue  with  the  application  is  the  lack  of  tracking 
of  the  other  team  members  on  the  mission.  The  initial 
screen  shows  the  other  team  members  but  they  are  not 
tracked  on  the  screen.  This  issue  should  be  improved  in 
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future  versions  of  the  software,  as  the  precise  location  of 
all  team  members  is  critical  for  safety  and  security 
reasons . 


Figure  46.  Second  Test  Location 


Overall,  the  tracking  tests  demonstrated  the  power  and 
precision  of  the  SPARCCS  application  in  a  long  duration 
vehicle-tracking  mission.  Through  precise  tracking,  the 
SPARCCS  application  can  provide  valuable  information  to  a 
command  and  control  station,  improving  situational 
awareness  for  all  mission  members. 
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C .  TESTING  CONCLUSION 

The  controlled  and  field  tests  demonstrated  the 
applicability  and  viability  of  the  SPARCCS  application  in  a 
variety  of  scenarios.  From  urban  campuses  to  remote  sites 
high  in  the  mountains,  the  SPARCCS  application  functions 
properly  to  deliver  data  and  photographs  as  well  as  GPS 
maps  and  precision  locations  to  users  to  improve 
situational  awareness.  Overall,  SPARCCS  performs  well 
depending  on  the  strength  of  the  wireless  signal.  The 
source  of  the  signal,  in  terms  of  a  specific  device,  does 
not  matter  as  long  as  the  device  produces  a  viable  WiFi 
cloud.  The  next  chapter  summarizes  the  testing  process  and 
provides  clear  recommendations  for  the  use  of  the  SPARCCS 
application  as  well  as  future  recommendations  for  the 
testing  of  the  system.  It  also  provides  a  summary  of 
conclusions  for  the  testing  program. 


104 


V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  TEST  CONCLUSIONS  AND  RECOMMENDATIONS 

The  testing  in  this  thesis  brought  up  critical  points 
regarding  the  use  and  development  of  the  SPARCCS 
application.  It  can  be  concluded  that  the  application  is  a 
viable  concept  that  applies  to  civilian  emergency  services 
as  well  as  the  military.  The  application's  concept  could 

even  be  applied  to  academia  and  the  general  public,  such  as 
for  field  exploration,  scientific  wilderness  missions,  or 
recreation  with  large  groups. 

The  SPARCCS  application  already  shows  promise  as  a 

robust  application,  even  at  this  stage  of  development.  It 
works  well  in  a  variety  of  scenarios  and  works  on  all  of 
the  wireless  platforms  tested.  The  mobile  application 
functions  equally  well  on  smart  -as  well  as  all  sizes  of 
tablet  PCs.  The  browser-based  Internet  application  worked 
well  on  all  evaluated  computing  devices,  including  Apple 

and  PC  computers  and  various  tablets. 

The  SPARCCS  application  is  still  in  development.  As 

such,  the  testing  was  designed  to  find  issues  and  errors 
that  can  be  rectified  in  future  versions  of  the  software. 
The  testing  succeeded  in  this  endeavor.  The  testing  also 
demonstrated  the  benefits  and  limitations  of  various 
wireless  technologies  that  SPARCCS  may  employ. 

Table  19  takes  the  findings  and  conclusions  of  the 
testing  and  presents  a  set  of  recommendations  for  future 
use  and  development  of  the  SPARCCS  application.  It  provides 
some  key  recommendations  gleaned  from  the  testing  of  the 
SPARCCS  application  and  associated  wireless  devices. 
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Table  19.  SPARCCS  Recommendations 


Number 

Recommendations 

1 

Utilize  SPARCCS  on  the  Android  OS  of  2.3  to  4.0.3. 

2 

Utilize  the  SPARCCS  Internet  application  on 

networks  that  do  not  block  the  Google  Appspot 

applications.  Military  networks  such  as  the 

"army.mil"  network  block  such  applications.  This 

should  be  considered  before  the  mission  commences. 

3 

Utilize  SPARCCS  on  the  proper  wireless  platform  to 

optimize  application  performance. 

4 

For  campus  settings,  cellular  wireless  may  be  more 

optimal  than  BGAN. 

5 

For  wildland  settings,  cellular  wireless  may  be 

optimal  if  there  is  a  signal. 

6 

For  flat  unobstructed  locations,  the  BGAN  may 

provide  the  best  distance  performance. 

7 

For  remote  locations,  the  BGAN  is  optimal  and 

should  be  utilized. 

8 

For  long  range  network  bridging.  Wave  Relay  devices 

outperform  WiMax  devices,  so  they  should  be  the 

first  choice  in  equipment. 

9 

Wave  Relay  devices  provide  superior  hotspot 

coverage,  but  can  only  be  used  if  a  120  volt  power 

source  is  available,  or  a  vehicle  inverter,  or  a 

Persistent  Systems  power  enabled  backpack  is  used. 

Therefore  planning  should  accommodate  this  power 

requirement . 
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Number 

Recommendations 

10 

BGAN  units  provide  longer  service  than  Jetpacks  by 

over  3  hours  so  for  longer  operations  the  BGAN 

should  be  utilized  if  practical. 

11 

The  Jetpack  provides  excellent  support  for  long 

range,  continual  tracking  and  should  be  used  for 

long  range  tracking  missions  if  service  is 

available  on  the  route. 

12 

The  BGAN  has  an  optimal  range  of  a  1100  ft  radius 

on  a  flat  area  with  no  obstructions  and  should  be 

the  first  choice  for  this  mission  scenario. 

13 

The  Jetpack  has  an  optimal  range  of  700  ft  with  or 

without  obstructions  and  should  be  used  if 

buildings  or  shrubbery  are  in  the  mission  area. 

14 

Both  the  Wave  Relays  and  the  WiMax  devices  can  form 

long-range  networks  of  over  9  miles  and  should  be 

used  for  such  distances  if  the  mission  requires 

long-range  networking. 

15 

The  BGAN  antenna  requires  near  line  of  sight  for 

optimal  signal  reception,  so  mission  planning  must 

accommodate  this  issue. 

16 

The  Jetpack  does  not  require  line  of  sight  and  may 

be  more  optimal  than  the  BGAN  in  congested  urban  or 

wildland  settings. 

17 

WiMax  bridges  require  line  of  sight.  Wave  Relay 

bridges  do  not.  This  must  be  considered  in  mission 

planning . 
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Number 


18 


19 


20 


21 


22 


23 


24 


Recommendations 

Wave  Relay  hotspots  do  not  require  line  of  sight, 
which  should  be  considered  when  planning  mission 
device  requirements. 

The  SPARCCS  application  is  fully  functional  with  a 
wireless  signal  of  at  least  150  Kbps.  Mission 
planners  must  take  this  into  consideration  when 
planning  missions  and  wireless  device  usage. 

Signal  decay  has  the  following  signs  on  the  SPARCCS 
application  which  should  be  noted  by  all  users  of 
the  system  to  be  able  to  diagnose  this  issue: 
application  halting,  application  freezing, 
incomplete  interface  painting,  inability  to 
transfer  photos  or  data,  improper  navigation. 

SPARCCS  can  be  used  equally  well  on  smart-phone  or 
tablet  PCs.  Optimally,  the  7"  should  be  used  for 
ease  of  handling,  screen  visibility,  and 
navigability  of  interface. 

The  mission  creation  function  should  be  re¬ 
engineered  to  eliminate  navigation  errors  in  the 
application . 

The  team  member  icons  should  have  the  ability  to  be 
tracked  on  other  team  member's  maps.  This  should  be 
programmed  into  future  versions. 

Additional  space  for  text  should  be  provided  for 
all  text  acquisition  functions  such  as  mission 
creation  functions,  POI  functions,  image 
descriptions,  etc.  to  increase  data  collection 
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Number 

Recommendations 

capabilities . 

25 

The  DORCCS  acronym  should  be  replaced  with  the 

SPARCCS  acronym  in  all  text  message  boxes. 

B.  RECOMMENDED  FUTURE  TESTING 

This  testing  of  SPARCCS  was  only  the  initial  testing 
for  the  application.  As  the  application  matures,  further 
and  more  advanced  testing  will  be  required.  The  following 
are  recommendations  for  future  testing. 

1.  Human  Factors  Testing 

The  SPARCCS  application  runs  on  both  hand-held  devices 
and  tablets.  The  Internet  application  runs  on  any  device 
that  has  Internet  connectivity  and  a  supported  browser.  The 
implementation  of  human  factors  engineering  is  crucial  for 
an  optimized  application,  as  in  the  field  users  cannot  be 
burdened  with  navigation  issues,  color  issues,  lighting 
issues,  and  other  issues  of  interactive  functionality. 

A  complete  human  factors  testing  program  would  ensure 
that  both  applications  have  user  interfaces  that  are 
tailored  to  the  human  body,  such  as  the  location  of  buttons 
on  the  screen  or  colors  used.  Human  factors  engineering 
helps  optimize  applications,  and  as  such,  would  help 
maximize  the  potential  of  the  SPARCCS  application  for  a 
wide  range  of  missions  and  functions  in  the  real  world. 

Since  SPARCCS  is  a  field  application,  many  missions 
may  have  users  wearing  gloves,  eye  protection,  or  other 
bodily  protection.  An  un-optimized  SPARCCS  application  may 
compound  the  usability  issues  by  which  the  users  are 
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already  challenged.  This  is  why  human  factors  testing  is 
critical  to  the  validation  of  long  term  usability  and 
performance  of  the  SPARCCS  application. 

2 .  BGAN  on  the  Move  Testing 

The  BGAN  device  was  tested  in  a  stationary  setting;  it 
does  not  function  well  in  a  moving  vehicle  due  to  satellite 
tracking  issues.  The  BGAN-on-the-Move  is  a  device  that 
rests  on  the  top  of  a  vehicle,  attached  by  a  magnet,  which 
continually  tracks  the  INMARSAT  satellite  and  can  provide 
an  Internet  cloud  for  a  convoy  of  vehicles  over  the  range 
of  several  hundred  feet. 

Future  testing  of  SPARCCS  should  utilize  the  mobile- 
BGAN  device,  as  remote  locations  may  not  have  the  cellular 
signal  capabilities  that  were  tested  in  this  thesis.  Remote 
testing  over  long  distances  of  the  BGAN-on-the-Move  device 
would  add  another  communications  option  to  a  SPARCCS 
mission  application. 

3.  Cellular  Vendor  Tests 

The  Verizon  Jetpack  service  was  tested  throughout  this 
thesis.  However,  there  are  several  other  vendors  that  have 
the  same  or  similar  service.  Future  testing  should  include 
a  similar  assessment  of  various  vendors'  services  and  the 
performance  of  SPARCCS  on  that  service.  Since  SPARCCS  is  an 
application  that  can  be  used  across  the  country  by  police 
and  fire  services  as  well  as  military,  cellular  service  may 
vary  from  jurisdiction  to  jurisdiction.  By  having  data  on 
the  other  cellular  services,  users  of  the  application  can 
plan  for  use  at  incidents  with  the  optimal  service  at  the 
location  of  the  incident. 
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4. 


Wave  Relay  Mission  Packs 


One  of  the  major  limitations  of  the  Wave  Relay  system, 
as  tested  in  this  thesis,  was  the  power  requirement  for  the 
radios.  In  essence,  the  radios  must  be  tethered  to  a 
location  or  a  vehicle  with  an  inverter  in  order  to 
function.  The  manufacturer.  Persistent  Systems,  has  a  field 
pack  for  the  devices  that  consists  of  a  vest  system  with  a 
battery  pack  and  a  carrier  for  the  radios  with  antenna 
slots.  By  using  this  pack,  the  Wave  Relay  system  is  mobile 
and  individual  members  of  the  SPARCCS  mission  would  have 
their  own  radios  that  could  act  as  a  bridge  or  a  hot  spot. 
In  essence,  a  human-hosted,  ad  hoc,  wireless  mesh  network 
would  be  able  to  be  constructed  to  improve  wireless 
connectivity  for  the  mission  team,  which  would  greatly 
expand  the  range  of  the  SPARCCS  application. 
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APPENDIX  A.  SAMPLE  TEST  DATA 


A .  CONTROLLED  TESTS 


Table  20.  Wireless  Device  Control  Tests 


Device 

Date 

Location 

Personnel 

Notes 

BGAN 

Jun  8 

2012 

NPS 

Dulo,  HFN 

The  BGAN  device  was 

tested  at  the  NPS  campus 

to  ensure  proper 

functionality.  The 

device  failed  at  first 

but  it  was  noted  that  a 

tree  branch, 

approximately  20  feet 

from  the  device.  Figure 

11  was  within  the  line 

of  sight  of  the  device 

and  the  Inmarsat 

satellite.  Shifting  the 

device  over  3  feet 

alleviated  this  problem. 

The  system  checked  out 

as  fully  functional 

BGAN 

Jun 

14 

2012 

CalFire 

Dulo,  HFN, 

CalFire 

The  BGAN  device  was 

tested  with  several 

contingencies.  The  sky 

was  completely  overcast 

but  the  device  connected 

properly  with  the 

satellite.  The  device 
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Device 

Date 

Location 

Personnel 

Notes 

did  not  function  through 

a  window  screen.  The 

device  did  connect  with 

the  satellite  through  a 

window  that  did  not  have 

a  screen.  The  device  did 

not  connect  to  the 

satellite  when  a 

building  blocked  part  of 

the  line  of  sight  with 

the  satellite.  The 

device  experienced 

connectivity  problems 

when  being  pointed  at 

the  satellite  through  a 

wire  fence.  The  optimal 

method  to  acquire  strong 

satellite  connectivity 

was  determined  to  be  the 

use  of  the  audio 

strength  signal.  Using 

the  visual  strength  bars 

on  the  device  display 

provided  less  precision 

in  device  pointing. 

WR, 

Jun 

CalFire 

Dulo,  HFN, 

The  wave  relay  radios 

WiMax 

21 

2012 

CalFire 

were  set  up  in  the 

classroom  and  in  the 

extended  parking  lot  of 

the  CalFire  campus.  The 
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Device 

Date 

Location 

Personnel 

Notes 

radios  were  able  to 

operate  in  bridge  mode 

as  well  as  wireless  node 

mode.  Four  wave  relay 

radios  were  tested  and 

connected  into  a 

wireless  mesh  network 

and  were  all  able  to 

communicate  with  each 

other  properly.  No 

communications  issues 

were  noted.  The  only 

issue  with  the  devices 

was  that  they  required 

120  volt  power  and  as 

such  mobility  in  testing 

the  devices  was  highly 

limited  to  the  length  of 

the  power  cords  and  the 

availability  of  wall 

power  outlets.  Omni 

directional  antennae 

were  used  and  thus  line 

of  sight  was  not 

required  for  the  radios. 

All  worked  well  in 

bridge  mode  and  as 

wireless  hotspots.  The 

BGAN  device  was  used  as 

the  Internet 
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Device 

Date 

Location 

Personnel 

Notes 

connectivity  device  and 

connected  well  with  the 

wave  relay  radios. 

The  WiMax  devices  were 

tested  in  the  classroom 

and  in  the  parking  lot. 

Point  to  point  bridge 

communication  between 

devices  was  established 

and  a  small  mesh  network 

was  established  with 

good  communications. 

Again,  power  cord 

mobility  was  an  issue 

with  the  WiMax  devices. 

The  devices  also 

required  strict  line  of 

sight  with  the  other 

devices  in  the  network. 

The  BGAN  device  was  used 

as  the  Internet 

connectivity  device  and 

connected  well  with  the 

WiMax  broadband  bridges. 

Jetpac 

Jun 

Outdoor 

Dulo 

The  outdoor  test  range. 

k, 

BGAN 

23  to 

27 

2012 

Test  Lab 

Figure  12,  was  used  for 

the  final  device  tests. 

A  test  range  of  200  feet 
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Device 

Date 

Location 

Personnel 

Notes 

was  set  up  in  the 

outdoor  test  range  to 

ensure  that  the  devices 

worked  well  with  the 

Android  systems.  The 

Jetpack  functioned  well 

at  a  signal  strength  of 

4  bars  and  exchanged 

viable  data  at  all 

points  from  0  feet 

through  200  ft  at  25 

foot  increments.  The 

BGAN  device.  Figure  13, 

connected  properly  to 

the  satellite  despite 

the  fact  that  tree  lines 

were  in  the  line  of 

sight.  The  use  of  an 

external  compass  was 

introduced  in  this  test. 

The  BGAN  device  has  a 

design  flaw,  where  the 

aiming  compass  is  on  the 

bottom  of  the  device 

making  it  obscured  when 

the  device  is  in  use.  An 

external  compass 

assisted  well  in  the 

pointing  of  the  device 

and  made  satellite 
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Device 

Date 

Location 

Personnel 

Notes 

acquisition  more  rapid 

and  accurate .  The  BGAN 

device  exchanged  viable 

data  as  demonstrated  at 

each  step  in  the  test 

range . 

The  wave  relay  devices 

were  not  tested  in  the 

outdoor  test  range  due 

to  the  fact  that  120 

volt  power  was  not 

available.  This  is  a 

distinct  issue  with  the 

wave  relay  devices  and 

one  that  needs  to  be 

addressed  before  the 

wave  relay  radio  is 

integrated  into  SPARCCS 

use . 

Table  21.  Basic  Application  Tests 


Test 

Device 

Results 

Account 

All  5 

User  accounts  were  created  with 

Creation 

mobile 

all  5  of  the  mobile  test  devices 

devices , 

on  the  mobile  application  as  well 

Toshiba 

as  through  the  Internet 

laptop 

application.  In  general  there 

were  no  problems  with  account 
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Test 

Device 

Results 

creation.  To  be  noted,  the 

acronym  DORCCS  continued  to 

appear  in  various  message  boxes. 

This  could  possibly  confuse  a 

user  and  thus  the  code  should  be 

reviewed  to  replace  all  instances 

with  the  SPARCCS  acronym.  On  the 

tablets  the  account  creation 

worked  fine  but  on  the  smaller 

smart  phones,  the  text  boxes  were 

small  and  it  was  difficult  to 

navigate  the  screen.  Several 

attempts  to  create  the  user 

accounts  had  to  be  taken  due  to 

the  difficulty  in  scrolling  up 

and  down  the  small  screen  which 

was  made  smaller  with  the  screen 

keypad.  All  accounts  appeared  on 

the  Internet  application 

precisely  as  inputted.  Creating 

the  accounts  on  the  Internet 

application  posed  no 

difficulties . 

Login 

All 

Both  applications  went  through 

login  testing  which  was  a  basic 

test  to  see  if  accounts  created 

were  viable  and  facilitated  login 

into  the  system.  All  devices  were 

able  to  login  to  the  system  on 

both  applications  well  and 
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Test 

Device 

Results 

without  incident.  One  note:  at  a 

low  signal,  the  SPARCCS  mobile 

login  screen  is  not  painted 

properly  on  the  screen.  This  was 

tested  thoroughly  and  it  appears 

that  at  0-1  bars,  this  issue 

arises.  With  1-5  bars  this  does 

not  arise.  Through  rigorous 

testing  of  this  issue  it  can  be 

confirmed  that  this  is  a  signal 

strength  issue  only,  not  a 

SPARCCS  application  issue.  This 

should  be  noted  in  SPARCCS 

troubleshooting  documentation. 

Mission 

Creation/ Join 

All 

The  mission  creation  function 

worked  successfully  on  both 

applications.  However,  in  some 

cases  it  was  difficult  to 

navigate  from  the  user  screen  to 

the  mission  creation  screen  and 

back  again.  For  example,  when  a 

mission  was  created  or  joined, 

the  back  navigation  brought  the 

user  back  to  the  mission 

creation/ j oin  screen  and  the  only 

option  was  to  go  back  into  the 

mission  creation  page.  To  solve 

this  issue  the  application  had  to 

be  restarted.  This  occurred 

usually  when  a  new  mission  was 
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Test 

Device 

Results 

created  and  then  a  user  tried  to 

join  the  mission  shortly  after. 

This  appears  to  be  a  design  issue 

in  the  application,  and  improved 

navigation  in  the  mission 

create/ join  page  should  solve 

this  issue. 

Point 

Interest 

of 

All 

The  point  of  interest  function 

was  tested  on  all  devices.  This 

function  worked  well  on  all 

devices  without  incident.  The 

POIs  created  appeared  on  the 

Internet  application  as  inputted 

and  the  Google  map  was  updated 

with  the  POI  symbol  at  the  exact 

point  of  the  POI . 

Photographic 

Evidence 

All 

The  photographic  function  was 

tested  on  all  devices.  In  each 

instance  a  photograph  was  taken 

and  inputted  in  the  system.  In 

all  cases  the  photograph  was 

successfully  transferred  and 

visible  in  the  Internet 

application.  The  photographs  were 

clear  and  of  high  quality.  The 

camera  icon  also  appeared  on  the 

Google  map  at  the  point  of  the 

image  acquisition.  No  issues  were 

discovered  after  extensive 
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Test 

Device 

Results 

testing  of  this  function. 

Google  Map 

Presentation 

All 

All  devices  were  tested  with 

their  GPS  capabilities  in 

relation  to  the  SPARCCS 

application.  All  instances  with 

the  Google  maps  presented  precise 

maps  with  the  current  user  icon 

of  SPARCCS  located  within  5  feet 

of  the  actual  location  of  the 

device.  In  all  cases,  the  Google 

map  was  refreshed  properly  when 

the  user  moved  location  with  the 

device.  No  issues  were  uncovered 

concerning  Google  maps  or  GPS 

issues . 
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MIRA  MARINA  CAMPUS 


Table  22.  MIRA  Jetpack  Data  in  Kbps 


Location 

# 

Upload 

Speed 

Download 

Speed 

Notes 

1 

714 

752 

About  5  foot  elevation 

2 

479 

613 

Elevated  line  of  sight 

3 

387 

344 

Back  midpoint  of  concrete 

building 

4 

311 

598 

Back  %  of  concrete  building 

5 

189 

111 

Distant  corner,  noticeable 

degrade  of  signal 

6 

850 

876 

Significant  improvement  in 

signal 

7 

853 

750 

Line  of  sight 

124 


8 

612 

662 

Line  of  sight 

9 

641 

651 

Line  of  sight 

10 

767 

521 

Midpoint  of  wood  building 

11 

1263 

2236 

Note:  next  to  building  antenna 

12 

517 

115 

Far  end  of  wood  building 

13 

412 

450 

Line  of  sight  metal  building 

14 

917 

924 

Line  of  sight  metal  building 

15 

458 

313 

Far  end  metal  building 

16 

501 

531 

Midpoint  far  side  metal  bldg 

17 

895 

1146 

Near  line  of  sight 

Table  23.  MIRA  BGAN  Data  in  Kbps 


Location 

# 

Upload 

Speed 

Download 

Speed 

Notes 

1 

536 

171 

About  5  foot  elevation 

2 

530 

179 

Elevated  line  of  sight 

3 

228 

176 

Back  midpoint  of  concrete 

building 

4 

0 

0 

Back  %  of  concrete  building 

No  signal  after  3  reading 

attempts 

5 

0 

0 

Distant  corner  concrete 

building.  3  reading  attempts. 

6 

0 

0 

Midpoint  of  concrete  bldg  far 

side.  3  reading  attempts. 

7 

288 

150 

Line  of  sight 

125 


Location 

# 

Upload 

Speed 

Download 

Speed 

Notes 

8 

372 

184 

Line  of  sight 

9 

201 

226 

Line  of  sight 

10 

401 

211 

Midpoint  of  wood  building 

11 

0 

0 

Note:  next  to  building  antenna. 

3  attempts . 

12 

0 

0 

Far  end  of  wood  building.  3 

attempts . 

13 

224 

179 

Line  of  sight  metal  building 

14 

258 

213 

Line  of  sight  metal  building 

15 

0 

0 

Far  end  metal  building.  3 

attempts . 

16 

0 

0 

Midpoint  far  side  metal  bldg.  3 

attempts . 

17 

343 

190 

Near  line  of  sight 

D.  MIRA  CHEWS  RIDGE  SITE 


Table  24.  MIRA  Chews  Ridge  BGAN  Data  in  Kbps 


Distance  from 

BGAN  (Ft) 

Upload  Speed 

Download 

Speed 

Notes 

0 

447 

200 

5003 

elevation 

ft 

50 

450 

193 

Flat 

sight 

line 

of 

100 

544 

181 

Flat 

sight 

line 

of 

126 


Distance  from 

BGAN  (Ft) 

Upload  Speed 

Download 

Speed 

Notes 

150 

481 

180 

Flat  line  of 

sight 

200 

517 

242 

Flat  line  of 

sight 

250 

572 

245 

Flat  line  of 

sight 

300 

360 

105 

Sharp 

increase  in 

elevation  by 

9  feet,  loss 

of  line  of 

sight 

350 

243 

88 

First  reading 

0,  second 

reading  as 

shown 

400 

0 

0 

3  reading 

attempts 

450 

0 

0 

5015  ft 

elevation.  3 

reading 

attempts 

500 

0 

0 

3  reading 

attempts 
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FT  ORD  CONCRETE  TEST  RANGE  FIELD  TESTS 


Table  25.  Ft  Ord  Jetpack  Concrete  Test  Range  Data  in  Kbps 


Distance  (Ft) 

Upload  Speed 

Download 

Speed 

Notes 

0 

740 

1179 

100 

760 

1751 

200 

1048 

1316 

300 

1008 

1367 

400 

1254 

1179 

500 

949 

440 

600 

602 

138 

700 

531 

135 

800 

124 

64 

First  attempt 
0,  second 
attempt  as 
indicated 

900 

0 

0 

3  attempts 

1000 

0 

0 

3  attempts 

Table  26.  FT  Ord  BGAN  Concrete  Test  Range  Data  in  Kbps 


Distance  (Ft) 

Upload  Speed 

Download 

Speed 

Notes 

0 

182 

220 

100 

195 

259 

200 

190 

200 

300 

174 

320 

400 

45 

201 

500 

394 

251 

600 

128 

94 

700 

477 

179 

800 

619 

167 

Very  strong 

gusts  of  wind 

900 

401 

206 

Very  strong 

gusts  of  wind 

1000 

444 

181 

1100 

202 

70 

1200 

86 

40 

First  attempt 
0,  second 
attempt  0, 
third  attempt 
as  indicated 

1300 

0 

0 

3  attempts 
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FT  ORD  WILDLAND  TEST  RANGE  FIELD  TESTS 


Table  27.  Ft.  Ord  BGAN  Wildland  Field  Test  Data  in  Kbps 


Distance  (Ft) 

Upload  Speed 

Download 

Speed 

Notes 

0 

280 

159 

Flat 

100 

291 

164 

Flat 

200 

306 

176 

Flat 

300 

381 

159 

Low  shrubs 

400 

323 

189 

Low  shrubs 

500 

320 

111 

Low  hanging 

trees 

approximately 

25  ft  medium 
density 

600 

171 

86 

Medium 

density 

700 

107 

90 

Begin  High 

density  trees 

800 

0 

0 

High  density 
trees,  3 
attempts 

900 

0 

0 

Same 

1000 

0 

0 

Same 

Table  28.  Ft  Ord  Jetpack  Wildland  Field  Test  Data  in  Kbps 


Distance  (Ft) 

Upload  Speed 

Download 

Speed 

Notes 

0 

622 

908 

Flat 

100 

734 

978 

Flat 

200 

886 

1120 

Flat 

300 

845 

1022 

Low  shrubs 

400 

901 

1108 

Low  shrubs 

500 

765 

843 

Low  hanging 

trees 

approximately 

25  ft  medium 
density 

600 

521 

767 

Medium 

density 

700 

112 

272 

High  density 

800 

104 

176 

High  density 

trees,  3 
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Distance  (Ft) 

Upload  Speed 

Download 

Speed 

Notes 

attempts 

900 

43 

82 

Same.  First 
attempt  0, 
second 

attempt  as 
indicated 

1000 

0 

0 

Same .  3 

attempts 
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APPENDIX  B 


QUAD  DIAGRAM 


<£cmd38> 

C*n*t  tot  th*  SKxfy  <* 

Mobto  Divkm  omi  CcvrifTnrtcoWom 


Smart  Phone  Assisted  Rapid  Communication  and  Control 

System 


V  fi  ~  ^  >;-• 

/~t  -N.S  ♦  "* 

Operational  Capability 

•Provides  wireless  network  capabilities  to  teams  in  tactical  and 
operational  situations 

•  Facilitates  centralized  data  capture  and  real  time  transmission  of 
data  and  images/video 

•  Facilitates  locally  controlled  aerial  photographic  and  scanning 
capabilities  with  real  time  feed  to  the  C2  center 

•  Facilitates  GPS  team  locations  in  real  time  to  C2  center 

•  Increases  scene  and  location  situational  awareness  to  decrease 
decision  cycles  and  improve  decision  making  capabilities 

Problem/Technical  Approach 

Problem 

•Troops  in  the  field  or  Emergency  personnel  in  the  field  need  an 
effective  way  to  capture  live  scene  based  information  and 
pictures  and  transmit  them  to  a  C2  center  to  promote  rapid 
decision  cycles  and  situational  awareness  in  a  tactical  and 
operational  environment 

Technical  Solution 

•Implement  a  wireless  smart  phone  network  among  small  teams 
■  Utilize  the  photographic  and  texting  capabilities  of  the  smart 
phones  and  small  tablet  PCs  to  capture  data 

•  Utilize  wireless  capabilities  such  as  BGAN  and  Wave  Relay  to 
provide  communications  capabilities 

•  Utilize  rotorcraft  UAVs  for  aerial  photographic  capabilities 

Milestones/Deliverables 

Milestones 

•Team  cloud  communication  development  and  testing 

■Centralized  server  and  team  cloud  communications  development  and 
testing 

■Rotorcraft  UAV  communications  development  and  testing 

•Full  system  integration,  verification,  validation  and  testing 

Deliverable 

•Field  deployable  smart  phone  assisted  rapid  communication  and  control 
system  with  full  rotorcraft  UAV  integration  and  optimized 
communications  platforms 

Donna  A  Dulo  Version  2 
16  JUl  2012 
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